Crawling, indexing and eligibility
Crawling fetches content. Indexing organizes it for discovery. Eligibility means a page meets the relevant system’s requirements; it does not mean that page will be selected.
How the concept fits together
- Step 1Public page and access controls
- Step 2Crawler can fetch
- Step 3Index and eligibility checks
- Step 4Possible retrieval, not guaranteed selection
What it means for GEO
For Google AI Overviews and AI Mode, a supporting page must be indexed and eligible for a Search snippet. Check access controls and readable public content before treating visibility as an editorial problem.
What not to infer
Google’s eligibility rule is not a universal rule for every assistant. Search permissions and training permissions may differ.
How it works
Access, indexing and selection are different gates. A crawler may be blocked by robots rules, a login requirement or hosting infrastructure. A fetched page may still not be indexed. For Google AI Overviews and AI Mode, the documented requirement for a supporting link is indexing and eligibility for a Search snippet; Google states that no additional technical requirement is needed. Even an eligible page may never be chosen for a particular answer. Other assistants have their own documented controls, so this Google rule should not be generalized to all products.
GEO example
Illustrative scenario, not observed assistant behavior or measured results.
Illustrative scenario: a pricing guide is public in a browser, but a CDN challenge prevents a crawler from fetching it. Rewriting the headline will not resolve that access problem. First investigate the blocked request, then inspect Google indexing and snippet eligibility if Google AI features are the target. For ChatGPT search, review OAI-SearchBot controls separately. After access is corrected, a citation is still not guaranteed: the user’s question and competing sources affect what is selected.
What you can do
- Check public access, crawler permissions and hosting-layer blocks before changing the copy.
- Use Google URL Inspection for Google indexing diagnostics and review snippet controls for the target page.
- Make important facts available as text, connect pages with useful internal links and keep structured data consistent with visible content.
What to measure, and what it cannot prove
Record access status, indexing status and observed answer inclusion as separate observations. A successful fetch is not an index check, and an index check is not a citation check. Google reports AI-feature traffic within the Web search type in Search Console; do not present that combined traffic as a dedicated citation-share metric.
Evidence and authoritative sources
- Google Search Central: AI features and your website
Provides Google’s eligibility requirements, SEO practices and Search Console reporting scope.
- OpenAI: Crawlers and independent search/training controls
Provides OpenAI-specific search crawler controls, which are separate from Google indexing.
The sources establish the documented concepts and product behavior. The publishing checklists, scenarios and operational measurement rules are GeckoCheck educational interpretations, not provider-endorsed ranking instructions.
Source: Google Search Central: AI features and your website · OpenAI: Crawlers and independent search/training controls
Period: Reviewed October 12, 2026. Living provider documentation; cited research published in 2023 and 2024 where applicable.
Sample: Conceptual explanation and illustrative scenario. No measured observations.
Limitation: GeckoCheck educational interpretation. Examples are fictional; formulas use declared operational definitions, not universal provider metrics. Source documentation does not guarantee citation or ranking outcomes.