Reading robots.txt, fetching the page without JavaScript, checking structured data…
An example report for angular.dev, re-checked daily. Enter your own site above to see how it compares, or compare Angular with other frameworks.
One problem stops assistants from reading or quoting this page. Fix it first.
263 words in the HTML, loaded in 0.1 s.
| Crawler | Used for | Type | This page |
|---|---|---|---|
| OAI-SearchBot | ChatGPT search | AI search | Allowed |
| GPTBot | OpenAI model training | Training | Allowed |
| ChatGPT-User | ChatGPT, when a user asks | User request | n/a |
| Claude-SearchBot | Claude search | AI search | Allowed |
| ClaudeBot | Anthropic model training | Training | Allowed |
| Claude-User | Claude, when a user asks | User request | Allowed |
| PerplexityBot | Perplexity search | AI search | Allowed |
| Perplexity-User | Perplexity, when a user asks | User request | n/a |
| Googlebot | Google Search, AI Overviews and AI Mode | AI search | Allowed |
| Google-Extended | Gemini training (not Search) | Training | Allowed |
| Applebot-Extended | Apple model training | Training | Allowed |
| CCBot | Open dataset used by many models | Training | Allowed |
Blocking training crawlers is a choice and doesn't affect AI search. What each crawler does
No structured data found.
Fix: Add JSON-LD: Organization on the home page, Article with author and dates on articles, Product or SoftwareApplication on product pages.
How to fix itNo machine-readable date found. Assistants use dates and authorship to judge whether a page is current and who stands behind it.
Fix: Show "Updated [date]" and the author, and add datePublished, dateModified and author to the Article JSON-LD.
How to fix itEvery AI search and answer crawler is allowed.
How to fix it263 words are in the HTML before any JavaScript runs.
How to fix itReturns HTTP 200 with no noindex or nosnippet rules.
How to fix it3 of 16 subheadings are phrased as questions.
“Home • Angular”
Found /sitemap.xml.
Found /llms.txt.
How to fix itIt scores 83 out of 100. One problem stops assistants from reading or quoting this page. Fix it first.
None. Its robots.txt lets in every AI crawler this check looks at.
It was last checked on October 30, 2026 and is re-checked once a day.
This check shows whether they can. Surfacepath Pro will ask them your customers' questions every week and show whether you're mentioned, cited, or replaced by a competitor.