Sitemaps
How to work with Sitemaps in DW10
A sitemap is used to ensure that search engines can easily discover and index the key pages of your website and the relationships between them. It guides the search engine crawlers to all the important content, including pages that might be hard to find through regular navigation, and helps to maintain or improve your site's visibility in search results.
When using DynamicWeb you will have an automatically generated XML sitemap which is accessed with /sitemap.xml on your URL and could look like:
Pages in sitemap
The default setting of new pages is to be shown in the sitemap if it’s published.
To either activate or deactivate the setting you need to:
- Select Edit from the context menu of the specific page
- Go to the Publication tab
- Enable or or disable Show in sitemap
Robots.txt
The robots.txt file is a text file that instructs web crawlers (robots) on how to crawl and index the site's pages, this is helpful in improving SEO.
In the Advanced tab of the website settings you will find a Robots.txt section with the following settings:
- Link to sitemap.xml, check to add a link to sitemap.xml in robots.txt
- Include products in sitemap.xml, check to add product URLs to the sitemap.xml
- Robots.txt, this is where you can set the value of requests to /robots.txt on hostnames that they can or cannot access e.g. "Disallow: /private/"
An example of a robots.txt with Link to sitemap.xml checked and not allowed access to /private/:
Product groups and products in sitemap
When including product groups and products to the sitemap, DynamicWeb will use the URL settings of the product list page to include the correct structure as they are published on the website.
First you need to include product groups:
- Select Edit from the context menu of the product list page and go to the SEO tab
- In the URL section:
- Select Ecommerce in the setting URL Provider, this enables more settings
- Configure the following settings e.g. select shop, language, what to include, path to groups etc.
Then you include products in your sitemap:
- Go to the Advanced tab of the website settings
- Enable the setting Include products in sitemap.xml
Variants in sitemap
It is also possible to include product variants to your sitemap. When you have selected the Ecommerce URL provider the setting Include variants in sitemap.xml can be enabled or disabled.
By default, we do not recommend including variants in sitemaps as in some cases the sitemap can become quite large, often containing duplicate content of all variant products.
