Reading robots.txt, fetching the page without JavaScript, checking structured data…
AI assistants can reach, read and quote this page. The remaining items are polish.
470 words in the HTML, loaded in 0.3 s.
| Crawler | Used for | Type | This page |
|---|---|---|---|
| OAI-SearchBot | ChatGPT search | AI search | Allowed |
| GPTBot | OpenAI model training | Training | Allowed |
| ChatGPT-User | ChatGPT, when a user asks | User request | n/a |
| Claude-SearchBot | Claude search | AI search | Allowed |
| ClaudeBot | Anthropic model training | Training | Allowed |
| Claude-User | Claude, when a user asks | User request | Allowed |
| PerplexityBot | Perplexity search | AI search | Allowed |
| Perplexity-User | Perplexity, when a user asks | User request | n/a |
| Googlebot | Google Search, AI Overviews and AI Mode | AI search | Allowed |
| Google-Extended | Gemini training (not Search) | Training | Allowed |
| Applebot-Extended | Apple model training | Training | Allowed |
| CCBot | Open dataset used by many models | Training | Allowed |
Blocking training crawlers is a choice and doesn't affect AI search. What each crawler does
No machine-readable date found. Assistants use dates and authorship to judge whether a page is current and who stands behind it.
Fix: Show "Updated [date]" and the author, and add datePublished, dateModified and author to the Article JSON-LD.
How to fix itNo robots.txt, so every crawler is allowed.
How to fix it470 words are in the HTML before any JavaScript runs.
How to fix itReturns HTTP 200 with no noindex or nosnippet rules.
How to fix itFound SoftwareApplication, WebSite, Organization.
How to fix it“Next.js by Vercel - The React Framework”
This check shows whether they can. Surfacepath Pro will ask them your customers' questions every week and show whether you're mentioned, cited, or replaced by a competitor.
Found /sitemap.xml.
Found /llms.txt.
How to fix it