Publishers / gamespot.com
Invisible to machines
0 of 14 AI agents get the full page at https://gamespot.com/. Checked 2026-09-25 09:33 UTC.
gamespot.com scores 1 out of 100 for machine visibility, number 89 of the 100 content publishers checked on 2026-09-25. Even a plain request carrying a Chrome user agent was refused (HTTP 403), so gamespot.com serves its pages only to clients that pass a full browser check. Every crawler below got the same treatment, which is why the score is so low. All 12 refusals happen at the HTTP level with no matching robots.txt rule, so a crawler cannot tell the block is deliberate. Googlebot, GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Bingbot, Meta-ExternalAgent, Amazonbot, Bytespider, CCBot received a bot-management challenge instead of the page. Googlebot itself is blocked: Bot challenge served (HTTP 403, cf-mitigated=challenge). The raw HTML is an app shell with 3 words; content appears only after JavaScript runs. There is no llms.txt.
youfull pagedegradedblockedunknown
Human
- Status
- 403 → https://www.gamespot.com/
- Title
- Just a moment...
- H1
- none
- Words
- 3
- Scripts
- 1 tags · 3 KB · app shell
- Meta robots
- noindex,nofollow
- Canonical
- none
- Server
- cloudflare
Machine
- robots.txt
- none (HTTP 403)
- llms.txt
- missing
- llms-full.txt
- missing
- X-Robots-Tag
- none
- Blocked
- GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Meta-ExternalAgent, Amazonbot, Bytespider, CCBot
- Degraded
- none
- Challenged
- Googlebot, GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Bingbot, Meta-ExternalAgent, Amazonbot, Bytespider, CCBot
Gaps
- highHumans cannot load this page eitherHTTP 403. Every other check is measured against a broken baseline.
- high12 of 14 AI agents are blockedAssistants asked about this page will guess, or cite someone else. Deliberate for a paywalled publisher. Otherwise this is the biggest gap on the list.
- mediumBot-management challenge served to crawlersGooglebot, GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Bingbot, Meta-ExternalAgent, Amazonbot, Bytespider, CCBot received an interstitial instead of the page. A robots.txt rule is an honest block. A challenge hides the page without saying so.
- lowNo robots.txtEvery crawler is allowed by default. Add one if you want any say in the matter.
- lowNo llms.txtOptional, but a short curated index at /llms.txt is the cheapest way to tell assistants what matters on this site.
- highGooglebot is blockedBot challenge served (HTTP 403, cf-mitigated=challenge)
Every agent
| Agent | Type | robots.txt | Fetch | Words | Result |
|---|---|---|---|---|---|
| Human (Chrome)You | Human | no file | 403 challenge200 ms | 3 | blocked HTTP 403 |
| GooglebotGoogle · Classic search index. Also feeds AI Overviews. | Search engine | no file | 403 challenge199 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| Google-ExtendedGoogle · Robots-only token controlling Gemini training use. Never fetches on its own. | AI training | no file | n/a | unknown Robots-only token; allowed by robots.txt | |
| GPTBotOpenAI · Training crawler for OpenAI models. | AI training | no file | 403 challenge200 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| OAI-SearchBotOpenAI · Indexes for ChatGPT search results and citations. | AI search index | no file | 403 challenge199 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| ChatGPT-UserOpenAI · Live fetch when a user asks ChatGPT about a page. | AI assistant (live fetch) | no file | 403 challenge199 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| ClaudeBotAnthropic · Training crawler for Claude models. | AI training | no file | 403 challenge192 ms | 0 | blocked Bot challenge served (HTTP 403) |
| Claude-SearchBotAnthropic · Indexes for Claude web search. | AI search index | no file | 403 challenge198 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| Claude-UserAnthropic · Live fetch when a user asks Claude about a page. | AI assistant (live fetch) | no file | 403 challenge198 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| PerplexityBotPerplexity | AI search index | no file | 403 challenge198 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| Perplexity-UserPerplexity | AI assistant (live fetch) | no file | 403 challenge198 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| BingbotMicrosoft · Bing index. Also feeds Copilot and ChatGPT search. | Search engine | no file | 403 challenge197 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| Applebot-ExtendedApple · Robots-only token for Apple Intelligence training. | AI training | no file | n/a | unknown Robots-only token; allowed by robots.txt | |
| Meta-ExternalAgentMeta | AI training | no file | 403 challenge197 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| AmazonbotAmazon · Alexa and Amazon AI. | AI training | no file | 403 challenge197 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
| BytespiderByteDance · Known for ignoring robots.txt. | AI training | no file | 403 challenge191 ms | 0 | blocked Bot challenge served (HTTP 403) |
| CCBotCommon Crawl · Open dataset used to train most LLMs. | AI training | no file | 403 challenge195 ms | 3 | blocked Bot challenge served (HTTP 403, cf-mitigated=challenge) |
Snapshot
Measured 2026-09-25 09:33 UTC from a residential connection. Sites change their rules often, and sites that verify crawler IPs answer differently to different networks. Run a live comparison to see what gamespot.com does right now, from Cloudflare's network.