How to make sure Google crawls your site completely
There’s no point investing in SEO if Google doesn’t index your site correctly. While this considered a minor part of SEO, you’d be surprised about the amount of indexation problems clients have. The good news is that issues are easily fixed. And you can follow this guide if you want to make sure that Google crawls your site completely.
Why isn’t Google crawling every page on my sitemap?
We’ve seen several examples where Google doesn’t completely crawl a sitemap. Remember that a sitemap is inclusive rather than exclusive. This means that there will be some pages on your sitemap that Google doesn’t crawl and some pages that aren’t on your sitemap that it does crawl.
There are a couple of reasons why Google won’t crawl a page on your sitemap. It is usually down to thin or duplicate content issues. If you have a doorway page or a duplicate page with very little unique content on it, Google simply won’t bother to crawl it.
Shouldn’t Google index every page on my site?
Google will never index every single page on your site or anyone else’s. It just isn’t going to happen and this is totally normal. Some pages just aren’t relevant and it isn’t worth Google or Bing or anyone else’s time. Focus on your most important pages and make them as good as they can be
Tips for making sure that every page in your sitemap gets crawled
There are a couple of things that you can do (or not do) to make sure that Google does crawl every page on your sitemap. These include the following points.
Make sure links are canonical and not redirects
Google wants to see the final destination of URLS, not be redirected around your site. So for landing pages in particular, make sure that you use the final URL and not a redirect.
Don’t create new sitemaps to get crawled
If some of your pages aren’t getting crawled, it can be tempting create several new sitemaps. Don’t. If your pages aren’t getting crawled in the first place, the problem is with your pages, not with your sitemap.
Stop creating thin content pages
If you are creating pages on your website with very little added value and a lot of regurgitated content, you’re not going to get crawled completely. Things like local service pages for different cities that are all but the same are a prime example of this.
Create proper 404 pages
By creating a proper 404 page that other pages get redirected to, you can stop the production of low quality doorway pages. There are several WordPress plugins that can help you do this.
Use the indexation report in Google Search Console
Google’s new indexation report in the Search Console helps you to identify indexation problems on your site. It will show you the reason why there was an error and how many pages are involved.
If you’re still having indexation problems, maybe it is time to let the experts have a look. Our team can help you identify any issues and resolve them quickly and effectively.
