Logfile generated by web-servers such as Nginx, Apache, etc contains information that the widely used Analytics tools i.e. Google Analytics won’t display. For example, Google Analytics won’t display the IP address of the bot or the user. Log files contain a timestamp, HTTP protocol, request type, URL, status code, IP address, etc. Analyzing the log… Continue reading Analyze Log Files online
Author: pramod
Pramod is the developer/founder of Website Crawler. He loves building web applications.
How to create XML Sitemap for any website?
As you might know, Website Crawler – On Page SEO Checker is capable of extracting URLs from sitemap and processing them. From now on, you can now create XML sitemaps for any website with WC. For those who don’t know, a sitemap is a file that contains all links the search bots should crawl and… Continue reading How to create XML Sitemap for any website?
Nofollow Links Checker
“rel” is one of the attributes supported by hyperlinks. Before Google introduced the “sponsored” value for the “rel” attribute this month, nofollow and dofollow were the commonly used values. Links using the rel=”nofollow”/”dofollow” attributes are widely known as nofollow or dofollow URLs. DF links pass page rank to linked documents. Searchbots follow these URLs. NF… Continue reading Nofollow Links Checker
How to find low quality content on a website for free?
Thin content is one of the major reasons why websites suffer from Google Panda algorithm penalty. For those who are not aware, Google’s Panda algorithm has been designed to lower the rankings of websites that have On-Page SEO issues. Thin content is one of several On-Page search engine optimization issues. The Panda algorithm is now… Continue reading How to find low quality content on a website for free?
Broken Links Checker: Find 404 links on website
When a page of a website is unreachable, the visitor will see an error message in the browser’s tab. These messages are reported because of the following reasons: The database server isn’t working: All nonstatic websites save data to the database. If the database server isn’t working, the page won’t won’t be able to get data… Continue reading Broken Links Checker: Find 404 links on website
Crawl sitemap links using Website Crawler
A sitemap, as you may already know, is the most important part of a website. It contains a list of links of a website and helps search engines in crawling/indexing pages which it may not find it. Today, I have good news for the users of Website Crawler. WC can now crawl the links it finds… Continue reading Crawl sitemap links using Website Crawler
Google de indexing your site? Learn how to find the root cause of this issue and fix it
Google dropping 1000s of pages from its index is a nightmare for bloggers, webmasters, developers, and online business owners. One of my sites has around 28000 pages. Google had indexed around 12000 pages of this site but in the last few months, it started dropping the pages from its index. If you’re following SEO news… Continue reading Google de indexing your site? Learn how to find the root cause of this issue and fix it