How to Setup and Optimize an XML Sitemap

Last Updated / Reviewed : January 7th, 2025
Execution Time : 0.5-4 hours

img1

Goal : To set up your sitemap for the first time, or to audit your existing sitemap.
Ideal Outcome : Your sitemap has been properly fetched by Google, contains your core URLs and is properly configured making it perfectly crawlable by search engines.
Pre-requisites or requirements :
■ You need to have access to your website’s Google Search Console property.
■ If you don’t have it set up yet, you can follow SOP 020.
■ If you are using WordPress on your website you will need to have the Yoast SEO Plugin installed and activated on your website.
■ If you don’t have Yoast on your WordPress website yet you can follow the chapter on installing Yoast by going to SOP 008.

Why this is important : Having a sitemap allows search engine crawlers to more easily find the content you want them to find, and to understand how important each page is.
Where this is done : In Google Spreadsheets, your WordPress Admin Panel, and Google Search Console.
When this is done : A sitemap should be included from the beginning of the website. If your sitemap is manually generated the ideal scenario would be updating your sitemap every time there is a new page that should be indexed.
Who does this : The person responsible for SEO or website management.

Included Resource

⦁ AVM Solution – Sitemap Designer Spreadsheet
Design your sitemap or audit your current one while making sure you all your sitemap settings are correctly defined.

img2

Designing your sitemap

⦁ Gather a list of all the URLs of your website.
⦁ If you already have a sitemap and are simply auditing the sitemap, you can use that sitemap.
⦁ If you are using WordPress on your website skip to this chapter.
⦁ If you are using a custom platform or not using a CMS at all, and don’t have a list of your URLs you should ask your web-developer or the company that developed your website for it.
⦁ Open the⦁ ⦁ ClickMinded – Sitemap Designer Spreadsheet included with this SOP → Click File → Make a copy…

img3

3. Paste your URLs in the Spreadsheet:

img4

4. For each URL, select if you would want to include it in the sitemap or not. If you choose not to include it you should select a reason why not. Mark it as “Yes” as long as it follows all of these guidelines:
Note : If you have a lot of URLs remember you can use the filters in Google Spreadsheets to speed up the process sometimes.

Example : If you want to remove all the pages on the subpath /personaldata/ you can just filter for those URLs and mark them all as “No”.

img5

⦁ The URL is not blocked by the robots.txt file.
⦁ If you don’t have a robots.txt file yet, you can follow⦁ ⦁ SOP 057.
⦁ If you are unsure if a given URL is blocked by the robots.txt you can follow the procedure covered in⦁ ⦁ SOP 057 (Chapter: Validating your Robots.txt).
⦁ There are not multiple URLs pointing towards duplicate content.

Example :
⦁ asiteaboutemojis.com/summer-sale-blue-emojis
⦁ asiteaboutemojis.com/summer-sale-blue-emojis?irrelevant_parameter
⦁ The URL is not pointing towards sensitive information.

Example : If you have internal documents such as pricing tables, customer information, internal procedures, etc. Make sure they are not on your sitemap.
Note : Also make sure those pages are not indexed and preferably password protected.

Note 2 : You can learn how to apply a “No-index” tag to a page here.
⦁ The URL is not an irrelevant system generated page or file.

⦁ Example : If the URL is for a system generated log file, configuration file, or other.
⦁ As a general rule of thumb, any URL that should not be indexed or that is not relevant for your users or to search engines should not be on the sitemap.
⦁ Remember : Removing a URL from the sitemap does not by itself remove it from search engines.
5. For each URL, that you’ve selected to include in your sitemap assign it a “Priority” and a “Frequency”:

⦁ Priority : A value from 0.1 – 1. It is relative to your other website pages:
⦁ 0.1 – Means low priority.
⦁ 1 – Means the highest priority.

To define priorities take a look at your URLs and decide which ones are more important for search engines and for your users. The default value is 0.5.
Note : Do not spend a lot of time on this process. If you are unsure you can also set all priorities to 0.5. If you have pages that clearly are higher in importance (e.g: “Homepage”, “Category page”, etc) you can assign the same high value to all of them.
Note 2 : There is no gain in setting all priorities to 1. All it will do is signal that all pages are equal in importance.

⦁ Frequency : Select the frequency you’d like spiders to crawl your URL.
⦁ Note: This is not a command that will be followed by spiders. This is merely a data-point that they will use and take into consideration.

⦁ Always : Only select this option if your URL changes each time it is accessed.

example: it displays the bitcoin value and is updated every second)

⦁ Never : Only select this option if your URL content is archived and will never be updated again. Typically you should not select this option.

⦁ Hourly, daily, weekly, monthly, yearly : If your content is updated within one of these intervals select the one that applies.

⦁ Note : You should avoid selecting “Yearly” even if that is the case. You should also avoid selecting “Hourly” for all URLs unless it really is the case that you update them hourly.

 Generating your sitemap

⦁ If you are using a popular CMS (Content Management System) such as Drupal, Joomla, or other. You will either have the functionality built-in(Drupal) or an extension available to generate the sitemap. This SOP covers the specific process for WordPress only.

⦁ If you have a custom website, or your website does not have a CMS:
⦁ Open your sitemap spreadsheet that you’ve created on the first chapter of this SOP. Filter all the URLs that you have selected that you want to appear on your sitemap.

img6

⦁ Open Google Chrome and head over to⦁ http://www.timestampgenerator.com/tools/xml-sitemap-from-list/

⦁ Start copying and pasting the URLs from your spreadsheet into the text box of the XML Sitemap generator, according to the “Priorities” and “Frequencies” that are on your Sitemap Design. Click “Add to Queue” until you’ve added all your URLs according to their Priorities and Frequencies.

■ Note : Before clicking “Add to Queue” be sure to delete the list of URLs that will still be in the text input box, otherwise you’ll be adding them multiple times to your sitemap.

■ Example : Filter all the URLs that have both “Priority: 0.5” and “Frequency: Weekly” and copy that list to the tool above.

img7

Image – Step 1: Copying groups of URLs from your spreadsheet.

img8

Image – Step 2: Add your URL group to the queue.

⦁ When all your URLs have been added to the queue, scroll down and click “Create Sitemap”:

img9

⦁ Your browser will start downloading your XML sitemap.
⦁ The file will be named “sitemap.xml” if that is not the case, make sure you rename it to “sitemap.xml”.
⦁ You will now need to upload that XML file to the root of your domain.
■ Example: www.asiteaboutemojis.com/sitemap.xml
■ Note: If you don’t know or don’t have access to upload files to your server you can ask your web-developer or the company that developer your website to do so. In the next step there is an email template you can tweak for your needs.
⦁ If you have a “Robots.txt” file you should also add this line to your file:
Sitemap: http://www.yourdomain.com/sitemap.xml
■ Note : Make sure your replace “http://www.yourdomain.com” with your actual domain name.

■ Example :
Sitemap: http://www.asiteaboutemojis.com/sitemap.xml
■ Note 2 : If you are not sure if you have a “Robots.txt” file or if you don’t have one. You can follow SOP 057.
■ Note 3 : If you don’t know or don’t have access to upload files to your server you can ask your web-developer or the company that developer your website to do so. Here is a template you can use to send over email:
“Hi,
I’ve created a sitemap to improve the SEO of the website. You can find the file attached to this email. I need it to be placed in the root of the domain and it should be named “sitemap.xml”.
The “robots.txt” file located at the root of the domain should also be edited and this line should be added at the end:
“Sitemap: http://www.yourdomain.com/sitemap.xml”
Thank you.“

⦁ That’s it! If all went well you now have your sitemap ready to be submitted.
■ Remember : You should repeat this process frequently whenever you add more URLs to your website. If you add a lot of URLs constantly you might want to ask your web-developer or the company that developed your website to develop a solution that updates the sitemap dynamically to avoid too much manual work.
3. If you are using WordPress 
⦁ Install Yoast SEO Plugin by following SOP⦁ ⦁ this chapter of SOP 008.
⦁ On your WordPress Admin Panel’s sidebar hover “SEO” and click “XML Sitemaps”

img10

Note : If you don’t see this option, you will need to enable “Advanced features” first:
1. Click “SEO” → “Dashboard” → “Features”

img11

2. Click “SEO” → “Dashboard” → “Features”

img12

3. Click “Save changes”

img13

3. Make sure “XML Sitemap Functionality” is enabled:

img14

4. Click “User Sitemap” and enable “Author / user sitemap” if your website has multiple authors and you would like Google to easily index the content of those authors separately.

4. Click “User Sitemap” and enable “Author / user sitemap” if your website has multiple authors and you would like Google to easily index the content of those authors separately.

5. Click “Post Types”:

■ Posts : Keep in your sitemap unless you have determined you don’t want your blog posts on your sitemap.

■ Note : Typically you want them to be on your sitemap, so you should leave the default option as it is.

■ Pages : Keep in your sitemap unless you have determined you don’t want your blog posts on your sitemap.

■ Note : Typically you want them to be on your sitemap, so you should leave the default option as it is.

■ Media (attachment): Typically you will not want to have it in your sitemap. This does not mean search engines will not index your media. This refers to separate pages that WordPress generates every time new Media is uploaded.
■ Example : http://www.asiteaboutemojis.com/red-emojis/red-fire-emoji

6. Click “Excluded Posts” : If when designing your sitemap you detected pages or posts that should be excluded from your sitemap you should enter their ID here.

img15

■ Note : If you don’t know the ID of the blog post or page, on the sidebar click “Posts” (or “Pages” depending on the case) → “All Posts” → Click the blog post you want to exclude from your sitemap.

In your browser’s URL bar you’ll be able to identify the ID by finding it in the URL (the number followed by “post=”):

img16

7. Click “Taxonomies” : If you are using the “category” and “tag” features of WordPress on your pages and posts leave those options as default. If you are not using them, disable them.
8. Click “Save changes”

img17

Submitting your sitemap to Google Search Console

Note : To follow this chapter you need to have Google Search Console setup on your website. If you don’t have it yet. You can follow SOP 020 – Setting up Google Search Console.
⦁ Go to⦁ https://www.google.com/webmasters/tools;
⦁ Log in with your Google Account;
⦁ If you have more than one website, select your website’s property.

img18

⦁ On the sidebar select “Sitemaps”:
⦁ Note : If you’re still using the old version of Google Search Console click “Try the new Search Console”:

img19

img20

⦁ Under “Add a new sitemap” enter the name of the file of your sitemap (If you have followed this SOP and generated the Sitemap manually enter ‘sitemap.xml’, If you have generated the Sitemap through WordPress’s plugin Yoast SEO, enter ‘sitemap_index.xml’) after your domain name, and click “Submit”:

img21

⦁ Click “Got it”:

img22

⦁ That’s it, If everything went well you will see your sitemap was successfully fetched by Google:

img23

Note : If for any reason your sitemap wasn’t successfully submitted, you will be shown an error: