A sitemap.xml file is used to ensure that search engines can easily discover and index the key pages of your website and the relationships between them. It guides search engine crawlers to important content, including pages that might be hard to find through regular navigation, and helps maintain your site's visibility in search results.
When using Dynamicweb, you get an automatically generated XML sitemap at /sitemap.xml.
Dynamicweb generates sitemap.xml from URLs in the solution that are eligible for indexing - this includes:
- Pages in the page tree that are marked for sitemap inclusion
- Product groups that are marked for sitemap inclusion
- Products and variants that are active in published groups
Handling large sitemaps
For large websites, a single sitemap file is not sufficient. Dynamicweb handles this automatically.
When the number of URLs for a site exceeds 50,000:
- The sitemap is split into multiple sitemap files
- A sitemap index file is generated on the
/sitemap.xmlURL - Search engines access all sitemap files through the index
There is no upper limit on the total number of URLs a site can expose through sitemaps, as long as they are distributed across sitemap files via the sitemap index.
The generated sitemap files are stored in Assets under /files/system/sitemapxml/sitemap-{indexName}-XXX.xml and cached for 24 hours. The cache can be purged by deleting the files.
Multi-site, multi-language, and multi-domain sitemaps
Dynamicweb solutions often contain multiple websites and languages. How sitemaps are generated depends primarily on hostname configuration on websites:
- Hostnames define sitemap boundaries
- Areas using different hostnames generate separate sitemap.xml files
- Areas sharing the same hostname share a single sitemap.xml
- This ensures that each sitemap only contains URLs that belong to the same hostname
- When a solution contains multiple language areas hosted on the same domain
- All language URLs are included in the same sitemap
- URLs remain distinct and fully crawlable
- Sitemap splitting still applies per domain when needed
- In short, language grouping does not force separate sitemaps unless hostnames differ
- If a solution contains multiple websites with different hostnames
- Each hostname gets its own sitemap.xml
- Each sitemap can independently scale beyond 50,000 URLs
- Sitemap indexes are generated per hostname when required
- As such, there is no global sitemap index spanning multiple domains
Pages in sitemap.xml
The default setting for new pages is to be shown in the sitemap if they are published.
To change this setting:
- Select Edit from the context menu of the page
- Go to the Publication tab
- Enable or disable Show in sitemap
Product groups and products in sitemap.xml
When including product groups and products in the sitemap, Dynamicweb uses the URL settings of the product list page to include the correct structure as it is published on the website.
First include product groups:
- Select Edit from the context menu of the product list page and go to the SEO tab
- In the URL section, select Ecommerce in the URL Provider setting
- Configure the relevant URL settings, such as shop, language, what to include, and path to groups
Then include products in the sitemap:
- Go to the Advanced tab of the website settings
- Enable Include products in sitemap.xml
Product variants in sitemap.xml
It is also possible to include product variants in the sitemap. When you have selected the Ecommerce URL provider, the setting Include variants in sitemap.xml can be enabled or disabled.
By default, we do not recommend including variants in sitemaps because the sitemap can become very large and often contains duplicate content across variant products.
