{"schemaVersion":"ready-scan.resource-sources.v1","locale":"en","path":"/sources/source-0e8eaf1d95acd8d8","canonicalUrl":"https://readyscan.ai/sources/source-0e8eaf1d95acd8d8","machineUrl":"https://readyscan.ai/discovery/sources/source-0e8eaf1d95acd8d8.json","page":1,"pageSize":48,"totalCount":2,"totalPages":1,"previous":null,"next":null,"kind":"source","source":{"id":"source-0e8eaf1d95acd8d8","url":"https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler","title":"Does Anthropic crawl data from the web, and how can site owners block the crawler?","publisher":"Anthropic","publicationCount":2,"modifiedOn":"2026-10-10","path":"/sources/source-0e8eaf1d95acd8d8","canonicalUrl":"https://readyscan.ai/sources/source-0e8eaf1d95acd8d8","machineUrl":"https://readyscan.ai/discovery/sources/source-0e8eaf1d95acd8d8.json"},"items":[{"key":"claudeAiVisibility","citations":[{"id":"anthropic-crawlers","title":"Does Anthropic crawl data from the web, and how can site owners block the crawler?","publisher":"Anthropic","url":"https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler","accessedOn":"2026-07-27"}],"editions":[{"language":"en","title":"Separate Claude search visibility from ClaudeBot traffic","description":"Measure Claude mentions and citations with the correct search, retrieval, and training-agent context.","path":"/claude-ai-visibility","canonicalUrl":"https://readyscan.ai/claude-ai-visibility","machineUrl":"https://readyscan.ai/discovery/resources/claude-ai-visibility.json","modifiedOn":"2026-07-27"}],"selected":{"language":"en","title":"Separate Claude search visibility from ClaudeBot traffic","description":"Measure Claude mentions and citations with the correct search, retrieval, and training-agent context.","path":"/claude-ai-visibility","canonicalUrl":"https://readyscan.ai/claude-ai-visibility","machineUrl":"https://readyscan.ai/discovery/resources/claude-ai-visibility.json","modifiedOn":"2026-07-27"}},{"key":"insight-ai-crawler-search-training-controls","citations":[{"id":"anthropic-bots","title":"Does Anthropic crawl data from the web, and how can site owners block the crawler?","publisher":"Anthropic","url":"https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler","accessedOn":"2026-10-10"}],"editions":[{"language":"en","title":"AI crawler access is not one switch","description":"Separate controls for search discovery, possible model-training use and visits triggered by a user. The available controls and their effects differ by provider.","path":"/insights/ai-crawler-search-training-controls","canonicalUrl":"https://readyscan.ai/insights/ai-crawler-search-training-controls","machineUrl":"https://readyscan.ai/discovery/resources/ai-crawler-search-training-controls.json","modifiedOn":"2026-10-10"},{"language":"zh-Hant","title":"AI 爬蟲存取並非單一開關","description":"分開處理搜尋發現、可能用於模型訓練的內容，以及由用戶觸發的造訪。各供應商提供的控制及其效果各有不同。","path":"/zh/insights/ai-crawler-search-training-controls","canonicalUrl":"https://readyscan.ai/zh/insights/ai-crawler-search-training-controls","machineUrl":"https://readyscan.ai/zh/discovery/resources/ai-crawler-search-training-controls.json","modifiedOn":"2026-10-10"},{"language":"zh-Hans","title":"AI 爬虫访问并非单一开关","description":"分别处理搜索发现、可能用于模型训练的内容，以及由用户触发的访问。各供应商提供的控制及其效果各不相同。","path":"/zh-Hans/insights/ai-crawler-search-training-controls","canonicalUrl":"https://readyscan.ai/zh-Hans/insights/ai-crawler-search-training-controls","machineUrl":"https://readyscan.ai/zh-Hans/discovery/resources/ai-crawler-search-training-controls.json","modifiedOn":"2026-10-10"}],"selected":{"language":"en","title":"AI crawler access is not one switch","description":"Separate controls for search discovery, possible model-training use and visits triggered by a user. The available controls and their effects differ by provider.","path":"/insights/ai-crawler-search-training-controls","canonicalUrl":"https://readyscan.ai/insights/ai-crawler-search-training-controls","machineUrl":"https://readyscan.ai/discovery/resources/ai-crawler-search-training-controls.json","modifiedOn":"2026-10-10"}}],"versionId":"fnv1a64:dcca1844bd915ba4"}