AI crawler access
Whether your robots.txt blocks the crawlers behind ChatGPT, Claude, Perplexity, Google (Gemini and AI Overviews), Microsoft Copilot (Bing), Meta AI, Apple Intelligence and Amazon, and whether your sitemap, llms.txt and agent endpoints are in place.