As you might know, Website Crawler – On Page SEO Checker is capable of extracting URLs from sitemap and processing them. From now on, you can now create XML sitemaps for any website with WC. For those who don’t know, a sitemap is a file that contains all links the search bots should crawl and index. If your site is well structured, search bots will be able to find the pages. If your site is poorly structured and doesn’t have a sitemap, search bots may not crawl and index some of the pages on your website.
To make sure that the important pages of your site are indexed, you should submit the URL of the sitemap file. Where to save the sitemap.xml file? Well, you can save the file to any folder of your website’s directory.
WebsiteCrawler sitemap generator
Once Website Crawler crawls your website, head over to the projects section, and click the “XML Sitemap” option.
Now, you’ll see a form with several fields. In the first textbox, enter the number of URLs you want the sitemap to have. If you don’t want to see URLs containing specific words or characters in the XML file, enter the word in the textbox 2.
URLs in sitemaps can have priority. URL with higher priority might be crawled/indexed first. With WC, you can create multiple sitemaps. Each sitemap can have URLs with different priorities.
Once the sitemap is generated, you’ll see a URL. Click this link to see the XML file. Once the browser opens the file, right-click on the file’s data and select the “Save As” option.
Sometimes, you may update a page/post on your site. To make sure that bots learn about the change and index the newer version of the page, you must either add the modified date or changefreq to the sitemap. As of now, WC doesn’t support dates. However, it can create sitemaps with changefreq option. If you set the URL changefreq to “weekly”, bots will crawl the URL on a weekly basis. Likewise, setting changefreq to “Always” will make the search bot visit the URL often.
