Index Coverage Report: Metrics, How to Use, and Tips

What is an Index Coverage Report?

Index Coverage Report is a report feature provided by Google Search Console to provide information to webmasters about the indexing process by Google.

In addition, this report also displays the status of each page that has been visited or accessed by Google bots.

In its display, each page is grouped based on certain statuses. Whether it's pages that have been validated and added to the index, pages that are excluded from the index, pages with major problems, and also pages with warnings.

The Function of Index Coverage Report

In general, this report is here to help you monitor and understand the indexation status of each website page by Google. The complete functions are as follows:

• Monitoring the Indexing Status: Through this report, you can monitor the indexing status of website pages by Google.
• Evaluation of Excluded Pages: You can also use it to evaluate pages that are intentionally excluded from search engines, either due to policy or due to certain issues.
• Warnings and Error Messages: Apart from evaluating, this report will also provide warnings and error messages for pages that have indexation issues.
• Fixing Technical or Structural Issues: With this report, you can identify and fix technical or structural issues that are limiting the indexation of web pages.

How to Use Index Coverage Report

Here's how to use the Index Coverage Report to improve website performance and visibility:

• Log in to your Google Search Console account. Next, navigate to the menu on the left and choose "Pages" in the Index section.
• In the report view, you can choose what status you want to visualize on the graph, for example:
• All known pages: Displays all URLs found by Google, either through the sitemap or through external links.
• All submitted pages: Only URLs are included in the website sitemap. This allows you to focus on the pages that you actively include in the sitemap.
• Unsubmitted pages only: Only URLs that are not in your website sitemap but are found by Google through external links. 
• You can also find the Index Coverage metric, which is a set of data that provides an overview of the extent of how well your website is indexed by search engines. Here are the commonly used metrics:
• Valid Pages: The number of pages that are successfully indexed and available in search results.
• Submitted vs. Indexed Pages: This metric compares the number of pages submitted for indexing to the number of pages indexed.
• Excluded Pages: Provides information on the number of pages excluded from indexing, such as pages blocked by robots.txt or pages with the "noindex" directive.

Types of Index Coverage Report Issues

In the Index Coverage Report, several types of issues are commonly reported related to the indexation of pages on the website. Check out the details below:

1. URL Excluded by "No Index" Tag

When Google bots find a page but cannot index it due to the "No index" tag in the HTTP response, this can reduce the visibility and accessibility of your content in search results. 

2. URL Blocked by Google Page Removal Tool

This issue usually arises when you block a URL using Google's page removal tool. Keep in mind that this method is only temporarily effective. It takes 90 days for Google to show the URL in search results again. 

3. Server Error (Status Code 5xx)

You need to monitor this section regularly because Google will have difficulty indexing pages with server errors (Status Code 5xx).

If a server error occurs, you may need to contact the server administrator to fix it or check if the error was caused by an update to the site.

4. Redirect Error

A redirect error indicates that the redirect you set up was unsuccessful, so it does not transfer search engine bots and users from the old URL to the new one. 

This kind of error is usually triggered by poor redirect configuration, such as the use of redirect chains or looping. 

5. URL Blocked by Robots.txt

Blocking by robots.txt is also one of the problems in the Index Coverage Report.

If the website page is blocked by the robots.txt file, then you don't need to worry. Because the page can still be indexed in other ways (although the chances are small), especially if Google manages to find other information about the web page. 

If you want to make sure that the page is not indexed by Google, then remove the robots.txt blocking and use the 'noindex' command.

6. Blocked Due to Unauthorized Request (Status Code 401)

Indicates access to the page requires login with a valid user ID and password. 

To resolve this issue, you need to remove the authorization requirement or verify the Google bot to be able to access the page again.

7. Crawled - Currently Not Indexed

This indicates that Google bots have crawled the page URL, but are still considering whether the page should be indexed in search results or not. 

This could be due to various factors, such as content quality, relevance, or indexing settings.

8. Discovered - Currently Not Indexed

In some cases, this may indicate that Google will index it soon, or it could be related to an indexing budget issue where Google considers your website to be of low quality.

9. Alternative Pages With Correct Canonical Tags

You need to make sure that every alternative page on your site has the correct canonical tag to point Google to the main version of the page. 

The goal is to ensure that your content is indexed correctly and there is no duplication in search results.

10. User-Selected Non-Canonical Duplicates

You can use the URL inspection tool to find out which URLs are selected by Google as canonical. With the tool, you can easily identify the main version of the page accepted by Google as a reference.

11. Duplicates, Google Selects Different Canonicals From Users

When you select a page as canonical, Google sometimes selects another page as the canonical version, especially if the page you want as canonical is less internally connected than the non-canonical page.

12. Not Found (404)

A 404 error page shows that the requested page cannot be found because it has been changed or deleted. 

Therefore, you should prioritize resolving this issue as it can affect user experience and also damage your website's reputation.

13. Page with Redirect

A page with the status "Page with redirect" implies that the page has been redirected, so it is not yet indexed by Google. 

When you do a permanent redirect for a page, make sure you apply a 301 redirect to an alternative page that is still relevant.

14. Soft 404

The Soft 404 problem occurs when a website returns a 200 status code indicating that the page exists, but the content shows errors or is not relevant.

15. Blocked Due to Prohibited Access (403)

This status code 403 means that the server accepted the request, but refused to grant access permission. Here are the steps to deal with this issue:

• Grant access to anonymous visitors so that Google bots can access the URL.
• If unable to grant anonymous access, remove the URL from the sitemap.
• Use the noindex tag to prevent the URL from being indexed by search engines.