AI-friendly websites

Your site probably ranks fine on Google and returns an empty page to ChatGPT. Those are not the same crawler and they do not behave the same way.

The one technical fact that decides it

Googlebot and Bingbot execute JavaScript. GPTBot, ClaudeBot, PerplexityBot and Applebot-Extended do not — they read whatever HTML your server returned and stop.

A React or Vite single-page app returns <div id="root"></div> and paints the page afterwards. So the site is visible in Google AI Overviews and Microsoft Copilot, and completely absent from ChatGPT, Claude and Perplexity. Rankings hold, analytics look fine, and the loss is invisible — it shows up as never being mentioned when somebody asks an assistant for a shortlist in your category.

Test your own site in ten seconds

Fetch it the way an AI crawler does and look for your actual headline:

curl -A "GPTBot" -s https://yoursite.com/ | grep -i "<h1"

Nothing back, or a bare root div? That is exactly what the assistants see.

What actually fixes it

Prerendering or server-rendering the routes that matter, so a complete HTML document arrives on the first byte. The app still hydrates and behaves identically for people. Then: a sitemap generated from the router rather than maintained by hand, JSON-LD stating facts about you, headings phrased as the questions people ask, and a robots.txt that names the AI crawlers explicitly — Google-Extended in particular is opt-in by absence, so silence is a no.

What we will not oversell

llms.txt. It is the fashionable recommendation. The measured reality is that AI crawlers almost never fetch it, no study has found a link between having one and being cited, and Google has said it does not use it. We ship one anyway — it costs an hour and coding assistants genuinely read it. But anyone selling llms.txt as the thing that gets you into ChatGPT is selling you a file, not an outcome.