[This article is part of my guide to technical SEO which I recommend reading once you are finished]
Are you investing time and money in blogging for your business?
What would it feel like to put in all that effort in writing great content for your website, promoting it and getting people to your website only for them to see this as they look around your site.
“404 page not found”
It really affects your website credibility if visitors see this. It makes you look careless. You could have the most beautiful website with the most interesting content but if your visitor experiences one or more 404 errors, they will quickly get frustrated and leave. Never to return.
All that work you put in is wasted.
When I carry out technical SEO audits, I often see 404 errors.
In this article I will explain what they are and whether you should care. I will explain how they effect your SEO, how you can find them and what you can do to fix them.
Let’s get started with some 404 basics.
What is a 404 error?
A 404 error is an error code that is generated by your website server when your browser requests a file that the server cannot find. There are a lot error codes, you can see a detailed list of them here https://developer.mozilla.org/en-US/docs/Web/HTTP/Status
A 404 error is the one error code you are probably most familiar with because it is the one that you will see all over the web.
It’s the error you see when you follow a broken link. Sometimes you may hear it being called a broken link error.
There are two types of 404 error.
- Hard 404 error is the normal ‘page not found’ error returned by the server.
- Soft 404 error occurs when a page does exist so it return a webpage valid code of 200, but the page doesn’t contain the expected content, e.g. it is returning blank content.
Now you know what a 404 error is, before you can fix them you need to understand what is causing them.
What causes 404 page not found errors?
The 404 error can happen for a few reasons.
- There is a typo in the URL causing the browser to request a non existent URL.
- The page your browser is trying to reach has been deleted or moved but the URL on the website has not been redirected.
- Linking to an external site and they delete, rename or move their page and don’t inform you.
- The server hosting the files has become unreachable
- There is a problem with the domain DNS (it can’t be converted to an IP address or it doesn’t exist)
- The page has very little content, is blank or contains confusing information (for soft 404 error)
Whatever the cause, one thing is for sure, they can be incredibly frustrating to experience. You might think there is very little harm in an occasional 404 error, and you are right. On for two on a rare occasion won’t ruin you.
It’s when you have a more serious problem on your site causing repeated, multiple 404 errors that can cause your business serious damage by frustrating your website visitors and Google! Let me explain how.
How can 404 errors damage your business?
Having 404 errors can directly damage your business in the following ways;
It can effect your reputation. Instead of being known for a reputable, trustworthy business, if someone keeps finding missing pages on your website then their opinion will quickly change to one of a business who is careless and can’t be bothered to look after their site.
Less traffic to and around your site. People are trying to get to your site but they can’t. That could be a lot of traffic you are missing out on.
Reduced ROI on your marketing budget. With reduced traffic comes a reduced ROI you get from your marketing budget. A reduced marketing ROI means you will have to pay more to acquire a new customer which means your marketing budget may need to rise.
Not only can 404 errors damage your business directly but they can have an indirect effect on your business through SEO.
How 404 errors can damage your SEO?
The occasional 404 error isn’t likely to affect your SEO but if you have a large number of them then eventually the search engine spider is going to think you have a poor quality website and restrict the number of times it visits your website and how many pages it crawls when it does visit. Once a spider restricts itself then you can’t guarantee your best pages will be crawled. It can even take months for the crawler to index your new pages and that is bad news if you are investing in your blog. Eventually this will all impact your SEO.
You will know your SEO has been impacted because your traffic will start reducing gradually and your blogs will take a long while to show up in the search engines.
You will probably start putting even more effort in to your SEO and content in the hope that your traffic will pick up. That means your SEO investment will go up.
Hard 404 errors are more acceptable by Google so there is less chance of them directly impacting your SEO. But, indirectly, if people arrive at your site and can’t find what they are looking for they will bounce and that does affect your SEO.
Soft 404 errors are likely to hurt your SEO more because Google finds it harder to understand what is supposed to be on the page.
So depending on your priorities, you should be finding and eliminating soft 404 errors to keep Google happy, then finding and eliminated hard 404 errors to keep your visitors happy.
If only you knew about those pesky 404 errors earlier. You would be able to fix them and avoid these problems.
Identifying 404 errors is not complicated, in fact there are a few tools that can help you. Let me tell you what they are.
Identifying 404 errors on your website
If you care about your search traffic then you probably have Google search console installed. In the crawl error report, you can see if you have any 404 errors. Google search console reports both soft and hard SEO errors. In Google search console go to Coverage -> Excluded and don’t forget to look through all your data. By default you will only be shown 10 rows. To show more data go to the bottom of the page where is says ‘Rows per page’ and increase this or use the pagination arrows to scroll through your data.
If you see 404 errors in your search console report click on the error to reveal which pages have a problem. I will explain how to fix them a little further down this post.
In my data I have a few 404 errors, both hard 404 errors and soft 404 errors that I need to fix.
What to do if you don’t have Google search console?
First of all, go and install it. It’s free and easy to set up.
Next, if you don’t want to wait for the data to populate you can download a tool called Screaming frog, it’s one of my go-to SEO tools. It’s free but with a limit of 500 pages per crawl.
Crawl your site with screaming frog.
Then to find your 404 errors go to the tab that says response codes. Filter on 4xx codes to see your 404 codes. The screaming frog data isn’t as complete as the Google search console data.
You can also use the site audit feature of SEMRush, another one of my go to tried and tested SEO tools. But, similar to Screaming Frog, this tools doesn’t identify all your 404 errors.
Screaming Frog can only find 9 and it doesn’t specify hard 404 or soft 404 like Google search console. SEMRush could only identify 3.
I also tried an online tool called Brokenlinkcheck. This tool was quite slow but it has the ability to crawl a lot more pages than the free versions of Screaming Frog and SEMRush. It managed to find 84 broken links. Quite a lot higher than Google search console. When I looked at the errors they were quite different in nature to the ones that Google was telling me about. These links were ones that no one had clicked on and Google hadn’t tried to crawl (and so they didn’t appear in my search console)
I recommend using Google search console if you can then using this online tool to check for any links that Google search console didn’t know about.
So, by now, hopefully you have a list of 404 errors that your site is producing. It’t time to fix them.
How to fix your 404 errors
Before we move onto fixing your 404 errors I just need to say that you only need to worry about the 404 errors that indicate you have some sort of technical problem on your site. It is impossible to fix all 404 errors, they are just part of having a website. Some 404 errors will be created by people simply entering the wrong URL for your site. You can’t fix those and Google isn’t so hard on your site if you have 404 errors. The errors you need to fix are the ones that will give your users a bad experience, the ones you can control. These will harm your site, you visibility and eventually, your business.
First we will look at how to fix the hard 404 errors
These are the easiest ones to fix.
In your list of hard 404 errors, take a look at the URL. For each URL you need to decide which scenario is true;
1. The page exists and you want the page indexed and accessible
2. The page exists but you want the page hidden and redirected to another
3. The page has been deleted
Let’s tackle scenario 2 and 3. In both these scenarios you need to set up a 301 redirect.
What is a 301 redirect?
A redirect is a way of your website telling the browser that your page has moved, and instead of loading the original page it requested, you now want it to load another page. Remember those status codes from before, well, 301 is just another status code.
Why should you set up a 301 redirect
The longer you have a website, the more pages and posts you create. From time to Time you will change what you want to show on your website. You will delete pages and modify URLs. Old content can still be accessible to people even though you think you have deleted it. They can access it via search engines who might not be aware you have changed your content, and via links that have not been updated. When people access the old content they get a 404 page not found error. This causes a bad user experience and lot of frustration. Setting up redirects limits the frustration.
When should you set up a 301 redirect?
You need to set up a 301 redirect whenever you delete a page or post or some other content on your website. You need to redirect to the nearest, relatable piece of content. If you have nothing, then redirect to your home page.
You also need to set up 301 redirects if you change your URL structure. Let’s say your URL had ‘blog’ in it but you decided to move it. You will need to create a redirect for all the pages that had ‘blog’ in the URL so the right page gets loaded if someone found the old link.
How to set up a 301 redirect in WordPress
When you redirect a page, the page gives out a status code of 301, so you may hear redirects also being called 301 redirects. If you have a WordPress site you can do this with a plugin. There are several.
If you don’t want to use a plugin you can do this in your .htaccess file. This can be tricky to get right if you are not a coder so I will leave the detailed instructions for another blog post.
How to fix soft 404 errors
These errors take a little more brain power to fix. You have to try to work out why Google thinks there is a problem with them.
For each soft 404 error you need to decide which scenario is true.
- The URL is a text URL, that looks kind of familiar but it isn’t one that you want indexing.
- The URL is a text URL that you recognise from old content you removed.
- The URL is for content you want to keep.
- The URL looks unfamiliar, a bit like code, it contains letters, numbers, brackets.. lots of weird stuff that you don’t understand.
For scenario 1 and 2, then just redirect the URL as you would do for regular hard 404 errors.
For scenario 3, you need to work out why the content doesn’t look good to the search engine. Take a look at the page, is it as you expect? You need to find the error and fix it. Easier said than done.
For scenario 4, look at the URL. It might contain a hint as to what has caused it. You need that URL to return the correct status code. Most of the time it means giving it a 404 status which can be done using a plugin in WordPress.
Once you have fixed the error code you can request that Google crawls your URL again so it can correctly categorise your URL.
Making the most of 404 errors
So by now, you should have all your pages returning the correct codes and redirecting to the best content. But. You just can’t ignore the fact that sometimes people will see a 404 error on your site, through no fault of yours.
What do you do in this situation? What will keep your users happy?
Start by thinking 404 errors don’t need to be the end of the line for your users.
You should aim to keep them on your website and give them the value they came for. You can build a dedicated, unique 404 page to do that. One that helps them find what they are looking and. And, if you are strategic, it can help you get a few more leads for your business or grow your email list a little.
The websites I create have one such dedicated 404 page, and depending on your strategy, it can have a search feature, social media follow buttons, an email optin or a lead magnet download.
You should know enough about 404 errors to go and find them and fix some of them. It’s worth doing on a regular basis because the longer you leave it, the more errors build up. If you want help identifying and fixing crawl errors I do this as part of my SEO audit service. If you would like to find out more please email me with your questions.
Take action today
Go to Google search console and look for your 404 errors. If you have some, start to fix them. Let me know in the comments below if you have errors or need some help!
[This article is part of my guide to technical SEO which I recommend reading once you are finished]
SHARING IS CARING
If you have enjoyed this article and found value in it, please consider sharing it with someone who would benefit it. Not only will it help them but it will help me too. Thank you.