Indexing: how does your page end up in Google?

Indexing is the act of adding a page to a search engine's database. Only once a page is indexed can it appear in search results or be used by AI answers. If your page isn't in the index, it simply doesn't exist for search engines — however good the copy is.

This page explains how indexing works, why pages drop out and how to check and fix it.

Crawling, rendering, indexing, ranking

Four steps that often get conflated:

1. Crawling — a bot fetches the URL. If it can't (a block, an error code, a slow server), it stops here.
2. Rendering — the page is executed, JavaScript included, to see what is really there.
3. Indexing — the engine decides whether to store the page, which version is canonical and what the page is about.
4. Ranking — only then is it decided which queries the page competes for, and where.

So crawled is not indexed, and indexed is not ranking for your preferred term.

Why pages don't get indexed

• A noindex tag in the head or via an HTTP header.
• A block in robots.txt — the bot may not fetch the page.
• A canonical pointing elsewhere, so this variant is treated as a duplicate.
• Thin or duplicate content: pages that add nothing over existing ones.
• No internal links. A page that only lives in your sitemap gets little signal that it matters. See internal links.
• Error codes or redirect chains: 404s, 5xx or a series of hops.
• Content that appears only after JavaScript and cannot be rendered reliably.
• Crawl budget wasted on filter URLs, parameters and endless variants on large sites.

How to check indexing

1. URL Inspection in Google Search Console — the most reliable per-page source: is the URL indexed, which canonical did Google pick, when was it crawled?
2. The Pages (indexing) report — categories like "Crawled — currently not indexed" and "Alternate page with proper canonical tag" tell you exactly where it breaks.
3. Your sitemap — check it contains only indexable, canonical URLs.
4. Server logs — on large sites these show how crawl budget is actually spent.

Use site: queries as a rough indication only; they are incomplete and sometimes misleading.

What to do about it

• Make the choice explicit. Decide per page type whether it should be indexed. Filter, sort and internal search pages usually shouldn't.
• One canonical version per topic. Avoid several pages answering nearly the same thing; merge and redirect.
• Add internal links from relevant pages that are already indexed.
• Add real value. "Crawled — currently not indexed" is often a quality verdict, not a technical fault.
• Fix technical blocks: stray noindex tags, wrong robots rules, redirect chains.
• Keep speed and stability healthy, see Core Web Vitals.
• Add structured data where relevant: it helps understanding and presentation, not indexing itself.

Indexing and AI search

AI answers and AI search engines largely rely on the same foundation: content that can be fetched and understood. What helps indexing — clean HTML, clear structure, answers that can stand alone as quotes — helps there too. See GEO and llms.txt.

Frequently asked questions about indexing

Should every page be indexed?

No. Thank-you, filter and search pages usually shouldn't be. Fewer but stronger pages beats opening everything up.

What does "Crawled — currently not indexed" mean?

Google saw the page but didn't include it. Usually a signal about value or overlap with existing pages.

Does publishing more often help indexing?

Only if the content adds something. Volume without value increases the share of unindexed pages.

What should I do after a migration?

Set up 301 redirects per URL, refresh your sitemap and check indexing of your key pages in the following weeks.

Further reading

See SERP, structured data and long tail, or spar via /en/book-call.