Blog
Why your site might be invisible to AI
If your content only appears after JavaScript runs, AI answer engines that read raw HTML may see an empty shell. Here's the problem, how to spot it, and how to fix it.
Your site looks great in a browser. It might be nearly invisible to AI answer engines — and you'd never notice, because the tool you use to check it is the exact tool that hides the problem. The culprit is client-side rendering: when a page's content is built by JavaScript in the browser rather than delivered in the server's HTML response, a reader that doesn't run JavaScript sees almost nothing. Most AI crawlers, as of mid-2026, are exactly that kind of reader.
This is the specific problem we kept running into, and it's worth explaining plainly because it's so easy to miss.
The empty shell
Open the developer tools on many modern single-page apps and view the source — not the rendered DOM, the raw HTML the server sent. On a client-rendered site you'll often see something close to this:
<body>
<div id="root"></div>
<script src="/assets/bundle.js"></script>
</body>That's it. No headline, no body copy, no links — just an empty container and a script that will build the page once it runs. Your browser runs that script, so you see a full, polished page. A crawler that reads raw HTML and doesn't execute JavaScript sees an empty <div>. There is nothing there to read, extract, or cite.
Search engines like Google invested years in a rendering pipeline that executes JavaScript to work around this. Most AI answer engines are newer and lighter, and default to reading the HTML as delivered. So you can rank fine in Google — which rendered your page — while being invisible to an AI assistant that didn't. For the full picture of what these crawlers do and don't do, see how AI crawlers read your site.
How to tell if it's happening to you
Two quick checks, no tools required beyond a terminal:
- Curl the page. Run
curl -s https://yoursite.comand read the output. If your main heading and body text are in there, you're fine. If you see an empty root element and a pile of scripts with your content nowhere in the source, that content depends on client-side rendering. - Compare source to rendered. In the browser, "View Source" shows the raw HTML; the Elements panel shows the rendered DOM. If your content is in Elements but not in View Source, it was built by JavaScript — and a non-executing crawler won't see it.
Sight automates this: it flags pages whose visible text is a tiny fraction of the raw HTML — a reliable fingerprint of an unrendered app shell — and separately checks whether AI assistant crawlers like GPTBot, ClaudeBot, and PerplexityBot are even allowed to fetch your pages. Both have to be true for an AI engine to read you: the door has to be open, and there has to be something behind it.
The fix
You don't have to abandon React, Vue, or your framework of choice. The fix is to render the HTML on the server (SSR) or at build time (static generation) so the response already contains your content. The client-side framework then hydrates that HTML — attaching its interactivity to markup that's already there — instead of building the page from an empty shell. The user still gets a fast, interactive app; the difference is that the first HTML response is complete, so every reader, human or machine, sees your content.
Most modern frameworks support this directly, and static generation is often the simplest path for content that doesn't change per request. Whichever route you take, the test is the same one you started with: curl the URL and confirm your content is in the response.
If this sounds familiar, it should — it's the exact problem this project was built around, which is why one of the first things Sight does is check what a non-executing crawler actually receives.
Find out what AI crawlers really see on your site — run a free scan