Can AI crawlers render JavaScript? What GPTBot sees

If your site renders its content in the browser, most AI crawlers and LLM fetch tools see an empty page. That is the short answer to "can AI crawlers render JavaScript": with a few exceptions, they do not. This guide shows what they receive, how to check your own pages in a minute, and the framework-level fixes that make content readable without JavaScript.

What the evidence says

The most cited data comes from Vercel's analysis of crawler traffic across its network, published in December 2024. Its findings:

• None of the major AI crawlers it measured executed JavaScript. That list included OpenAI's GPTBot, OAI-SearchBot and ChatGPT-User, Anthropic's ClaudeBot, PerplexityBot, Meta-ExternalAgent, Bytespider and Common Crawl's CCBot.
• ChatGPT's and Claude's crawlers did download JavaScript files. They just did not run them.
• The exceptions were Gemini, which uses Googlebot's rendering infrastructure, and AppleBot, which renders through a browser-based crawler.

Crawler behaviour changes, and this is one company's measurement from late 2024. Check each operator's own documentation, such as OpenAI's crawler overview, for current user agents. The practical conclusion has not changed: content that exists only after client-side JavaScript runs is invisible to most AI fetchers.

See your page the way they do

A plain HTTP request is a close approximation of what a non-rendering crawler receives.

URL=https://example.com/
curl -sL -A "Mozilla/5.0 (compatible; render-check)" "$URL" -o raw.html
grep -oiE '<p[ >]' raw.html | wc -l
grep -o '<title>[^<]*' raw.html
grep -oE '<h[12][^>]*>[^<]+' raw.html | head

Then open view-source: on the same URL and compare it with what you see on screen. Missing headings, prices, descriptions or article text are the parts an AI crawler cannot read. The one-minute test covers the same checks from the reader's side.

Fix it in the framework

The fix is to put the main content in the first HTTP response. How depends on the framework:

Interactive parts, such as view counters, chat widgets and personalised blocks, can stay client-side. What matters is that the text, headings, links and metadata are in the HTML.

Why not dynamic rendering?

Dynamic rendering serves a prerendered snapshot to bots and the normal SPA to people. Google's documentation calls it "a workaround and not a long-term solution" and recommends server-side rendering, static rendering or hydration instead. It also adds a second rendering path that can drift from what visitors see. Use it only as a stopgap while you move to SSR or static generation.

Metadata that should always be server-rendered

Even pages that keep a client-rendered body should send these in the raw HTML:

• <title> and a meta description
• Open Graph tags for title, description and image
• A canonical link
• JSON-LD for the page type, such as Article, Product or FAQPage

The structured data guide explains how fetchers read these formats.

If you want the opposite

Some site owners want AI crawlers to stay away. The standard mechanism is robots.txt, with rules for each crawler's published user agent. RFC 9309 defines the format. It is a request that well-behaved crawlers honour, not an access control, so pair it with server-side measures if that matters to you.

Test it yourself

Add the curl check to your CI for your three most important pages: fail the build if the paragraph count drops below a threshold or the main heading is missing. That catches the most common regression, a refactor that moves content into a client-only component, before a crawler does.