Have you just created a new website or new piece of content and gone over to Google to search for it, only to find it’s not there yet?

You wait a day or two and search again… it’s still not there.

Where is it?

There are three questions that may spring to your mind…

Has Google visited my site yet and seen my content?

How often does Google visit my site?

How often does Google update its search results?

I’m going to help you answer all those questions in this blog post.

But first, let me quickly explain how the search process works.

How does your content get onto Google?

When you have a new site or a new piece of content you want to get onto Google you have two options.

    1. Do nothing. Eventually Google may find your site by following links from other places that point to your content.
      This method is risky because what if nothing ever points to your content?
      What if Google just takes ages to find your content?
  1. Tell Google. There are a few ways you can tell Google.
    1. If it’s a new site you can create a Google search console account and submit a sitemap.
    2. If it’s just new content you can update your site map or directly ask Google to crawl your URL.

 

Once Google knows about your content, or has followed a link to it, it ‘crawls’ you content.

This means Google looks at your content to see what’s there.

Google then makes a decision whether your content is good enough to be indexed. Indexed means it is in the big Google database, ready to be delivered to people who’s search query matches your content.

If you find your content or your whole website is not on Google then you need to identify the problem.

What stops you getting onto Google?

There are few problems that will stop you getting onto Google.

  1. Google couldn’t find your content and so couldn’t crawl it.
  2. Google found your content but couldn’t crawl it.
  3. Google crawled your content but didn’t index it.

 

When working out why you are not on Google, we need to start with 1. and work out if Google actually knows about your content.

How to verify that Google knows your website exists

Verifying that Google knows about your content helps answer the question ‘Has Google visited my site yet and seen my content?’

First you need to have an Google search console account set up. For the remainder of this article I will assume you have one and you have submitted a sitemap.

Quick tip: Before you start this check, please make sure your sitemap contains the content you want to appear on Google. it makes life a whole lot easier.

The first check you need to make is with your sitemap. From search console go to Sitemaps.

Where to find sitemaps in google search console

Google uses your sitemap to help it navigate around your site.

If Google has ever been to your site you will see your site map will have a date when Google last read it. You can see from mine below Google visited my site on the day I wrote this post.

Google search console sitemap report to see when Google last read the sitemap

 

You may find that the ‘last read’ data was a while ago. In this case you need to refresh your content more often so google knows to visit more often. If your site isn’t normally updated frequently, Google will not visit very often.

If you have nothing here at all then you need to get Google to read your sitemap. Do this by simply adding your sitemap to the search console and checking it was successful.

Once you have verified Google has visited your site, next you can verify that it has actually crawled the pages you want it to crawl.

How to verify Google has crawled your content

To find out which actual pages Google has crawled, click on the graph icon on the right hand side of the ‘Submitted sitemaps’ section.

How to see the pages google has crawled in search console

You will now be in the coverage report.

You will see some numbers that tell you how much of your sitemap has been crawled.

Click on the Valid block, then click on the ‘Submitted and indexed’ link to see your successfully indexed pages.

Look at the pages that have been crawled in google search console

 

Once you click on that link you will see a list of all the content that is in the sitemap and that has been correctly crawled and indexed.

You will also see the ‘last crawled’ date. Notice how Google doesn’t crawl all your pages at once. It crawls them in chunks.

If you don’t see your content in there then there may be a problem with it.

How to identify problems that may stop your content from being crawled and indexed

Earlier you clicked on’Valid’ to see what content had been correctly crawled and indexed, now I want you to click on ‘Excluded’ just to the right of it.

This will show you what content has been excluded and it will give you an indication of why.

Here are some of the excluded reasons you may come across.

Crawled – currently not indexed

This means Google has seen the page but decided not to index it.

On my website, I have a bunch of pages that I don’t want int search results for one reason or another.  Inside my website I have marked them as ‘noindex’.

If you find your page in this list but you want it indexed then check on your page you have not set a tag somewhere to ‘noindex’. This is often a setting in an SEO plugin if you are using WordPress. Just double check the setting as this is the most likely cause for this.

Discovered – currently not indexed

This means that Google has discovered your content (because it is in your sitemap) but it hasn’t crawled or indexed it yet.

I had a few of these in my search console report but when i went to inspect the URL, I was told the page was indexed so sometimes the report data is a bit old.

If your content is definitely not in Google then go to the URL inspection tool (top of the search console page) and enter URL. You will be told the URL is not on Google.

You may also see a reason why, such as ‘Not Found’.

Then you will see a link that says ‘REQUEST INDEXING’. Click on this to request to get the content indexed.

How to get an URL indexed Google search console

If after checking these steps you still can’t find your content then you might be blocking Google from crawling the content.

 

How to check if you are blocking Google from crawling your content

You should have a robots.txt file. This is a place where you put commands that tell bots what they can and can’t crawl on your site.

There is a chance you are blocking Google using this file.

For more information about how your robots.txt file read this Google article here

Conclusion

You should now know the steps to take to make sure your website and content gets into Google. Each time you create a piece of content, I recommend you give it a few days then check to see. If it is not listed in Google then find the reason why.

Do this regularly so you don’t just suddenly wonder why you have no traffic.

Fix the problems as they arise.

Take action

Check you Google search console data. Is everything you want to be found in Google listed in your sitemap?

Is everything in your sitemap being crawled and indexed?

Let me know in the comments below.

I do a sitemap and content check as part of my technical SEO audit. If you want help auditing your site, email me and I will help you out.

 

 

SHARING IS CARING

If you have enjoyed this article and found value in it, please consider sharing it with someone who would benefit it. Not only will it help them but it will help me too. Thank you. 

DON'T FORGET TO SUBSCRIBE TO THE BLOG!

Share This