How to Fix the Blocked Due to Other 4xx Issue in Google Search Console

This blog post will take you through the Blocked Due to Other 4xx Issue within Google Search Console. This is a widespread issue; if you have worked in SEO for any amount of time, you will have encountered it at least once.

We will use this article to discuss what the Blocked Due to Other 4xx Issue means, what causes this error to show up within your GSC account, where to find pages that are impacted by this issue, and how to solve the issue.

 

What Does the Blocked Due to Other 4xx Issue Mean?

The Blocked Due to Other 4xx Issue within Google Search Consols refers to situations where Google’s crawler, Googlebot, cannot access certain pages on your website because of a 4xx client error. That client error is not a 404 or a 410 error.

The 4xx family of status codes indicate client errors, where the issue is generally on the client’s side and not the side of Google. In the context of a website trying to get a page crawled for indexing and ranking, it almost always means a problem with the page’s availability or accessibility to Googlebot.

Here is an image of the Blocked Due to Other 4xx Issue appearing within Google Search Console:

What Causes the Blocked Due to Other 4xx Issue to Show in Google Search Console?

There are many different causes for the Blocked Due to Other 4xx Issue showing up in your GSC account or the GSC account of a client you are working on. Generally, the cause comes down to what particular 4xx code Googlebot receives when it tries to access a URL.

These break down into the following:

403 Forbidden

If your page sends a 403 Forbidden status code to Googlebot when it tries to access your page, it means the server is refusing to respond to the request. This can happen if access to pages is restricted, requires authentication, or is blocked for some reason.

401 Unauthorised

This is similar to the 403 error code. Still, a 401 Unauthorised error specifically indicates that authentication, such as a password, is required and has either not been provided or has been provided, but it is not sufficient. Googlebot cannot input access credentials for URLs, so it cannot access password-protected pages, for example.

422 Unprocessable Entity

A 422 Unprocessable Entity status code means that the server understands the request entity’s content type and that the request’s syntax is correct. Still, it was unable to process the contained instructions.

Essentially, the server says, “I understand what you’re asking and how you’re asking it, but I can’t do what you’re asking me to do.”

429 Too Many Requests

A 429 Too Many Requests status code appears when a user (or Googlebot) has sent too many requests in a given timeframe. This is also referred to as rate limiting and is seen most of the time on shared hosting platforms.

Other 4xx Codes

There are several other 4xx status codes which might be less common but can still cause pages to be blocked.

Whenever you encounter this issue within Google Search Console, it is crucial to investigate which specific 4xx is occurring on which pages. Once you have identified this, you can take the necessary steps to resolve the issue. We’ll discuss how to fix the issue later in the article.

Where to Find Instances of Blocked Due to Other 4xx Issues in Google Search Console

Luckily, finding your pages that have been impacted by the Blocked Due to Other 4xx Issue within Google Search Console is a straightforward task which does not take a lot of time.

First, head to your Google Search Console dashboard and click the Pages link underneath the Indexing section on the left-hand toolbar. We have included a highlighted screenshot below to indicate this:

Once you have access to your Page Indexing Report, after clicking on the above link, you should come to a page that looks similar to the screenshot below:

Scroll down on your Page Indexing Report, and you will be greeted with a list of why Google cannot crawl and/or index certain pages on your website. You will see any instances of the Blocked Due to Other 4xx Issue here.

We have highlighted the screenshot below so you can see this, too:

Click on this link, and you will be taken to a page highlighting only the pages impacted by the Blocked Due to Other 4xx Issue. At first glance, it will look like this:

Scroll down here, and you will be able to see all of the impacted URLs:

How to Fix Blocked Due to Other 4xx Issue Instances

To fix the Blocked Due to 4xx Issue/s appearing within your GSC dashboard, identify the specific 4xx error and then take the appropriate steps to resolve it.

Here’s a step-by-step approach to make things easier for you to understand:

Identify the Specific 4xx Error

We talked a little bit about this above, but the first step to solving the Blocked Due to Other 4xx Issue/s appearing on your URLs is to identify the specific 4xx issue that is occurring.

Head to your Page Indexing Report via Google Search Console and identify all URLs under the Blocked Due to Other 4xx Issue section. Once you have a list of these URLs, try to access them yourself using your browser or a tool like ScreamingFrog to crawl them.

The end goal here is that you have a list of URLs and a list of the 4xx status codes that appear for each when you try to access them.

Understand the Error

The next step is to understand the error being given for each URL.

If you see a 401 error, the request requires authentication. A 403 error means the server understands the request but refuses to authorise it. A 404 error means that the server cannot find the requested URL. A 410 error means that the resource requested is no longer available. Essentially, the page has been deleted. A 429 error means the user has sent too many requests in a specific timeframe.

We go through, in detail, the meanings of each 4xx error code earlier on in the article, so head to that section if you need more details or information.

Check and Fix Impacted URLs

You then need to check each URL and offer fixes.

For 404 and 410 errors appearing, if the pages are no longer needed, you can leave this alone or create redirects if the pages have any SEO value to them. If they were removed accidentally, restore them.

For 401 and 403 errors, you need to check your server settings and permissions. Ensure that Googlebot is not accidentally being blocked from crawling these URLs. Sometimes, servers have security protocols that can accidentally block search engine crawlers, so make sure this is not happening!

If you are seeing 429 errors, you need to adjust your rate-limiting settings and ensure that your website server can handle the number of requests Googlebot is sending to your website. You can use log file analysis technique to find the exact number of times Googlebot tries to access a specific URL in a given timeframe. This will help you understand what your rate-limiting settings need to be adjusted to.

Test the Fixes

After making the changes outlined above, you can use the URL inspection tool within Google Search Console to test whether Googlebot can now access your pages without issue or is still having trouble accessing specific URLs.

Submit URLs for Re-Indexing

Once the issues are fixed, and you have successfully tested them to ensure no other 4xx issues appear, you can submit the corrected URLs for re-indexing in Google Search Console.

Keep in mind, however, that Google Search Console has a limit of 10 per day. If you have more URLs than this, ensure the updated URLs are in your sitemap to give Google a better chance at getting your updated pages crawled quickly.

Monitor the Results

Keep a close eye on your Page Indexing Report within Google Search Console to see if the number of Blocked Due to Other 4xx Issues decreases over time. This should happen as Googlebot crawls and can access the URLs without issue.

By systematically addressing each issue as they appear, you can help ensure that Google can effectively crawl and index your website. This process is crucial for SEO and your website’s search visibility.

In conclusion, effectively addressing the Blocked Due to Other 4xx Issue in Google Search Console is crucial for ensuring that Google can crawl and index important pages on your website, a critical factor in maintaining and improving your website’s visibility in the SERPs. By understanding and methodically resolving each specific 4xx error on your site, you can enhance your website’s accessibility to Googlebot and optimise your site’s performance in search engine rankings.

Are you looking to use Google Search Console data to supercharge your SEO? Give SEOTesting a try! We have a range of valuable reports that you can use to find quick wins, analyse traffic increases/dips, improve content and much more. All use the data that is available in GSC to save you time.

Give us a try today and sign up for our 14-day free trial. No credit card required!