JavaScript Links Can Make Your Pages Invisible to AI Search Engines

Unlike traditional search engine crawlers, the bots of AI search engines and assistants lack the capability to execute client-side JavaScript. As a result, web pages with internal links built using JavaScript can become completely invisible in AI searches. A 41-day controlled experiment revealed that no AI crawler—including GPTBot, ClaudeBot, Bingbot, and PerplexityBot—was able to follow JavaScript-injected links.

Experiment Setup and Methodology

Conducted to obtain real-world data, the test utilized a business directory site featuring approximately 2,400 pages. Running across 1,062 pages within the site's hierarchical structure, the page links were divided into two groups:

• HTML Group (11 Sections): All sub-links were coded directly into raw HTML, allowing every bot to parse these links.
• JS Group (10 Sections): All sub-links were injected via JavaScript after the page loaded. A bot incapable of executing JavaScript would see no links in this group.

To avoid compromising the accuracy of the test, sitemaps were disabled, breadcrumb navigation was removed, and all alternative discovery paths were blocked. Consequently, the only way to reach the pages in the JS group was by processing the parent page's JavaScript code.

Surprising Results of the Controlled Experiment

Findings obtained from the log layer, which recorded bot requests over a 41-day period, clearly demonstrated that traditional SEO perception falls short in the era of artificial intelligence:

• Google’s crawler stack was the only entity that executed JavaScript and followed the injected links.
• However, almost none of this workload came from Googlebot, the primary crawler that builds the search index.
• Googlebot was only able to reach 2% of the pages linked via JavaScript.
• All other tracked AI bots—including GPTBot, ClaudeBot, Bingbot, PerplexityBot, Meta, and Amazon crawlers—yielded zero results from JavaScript-hidden links, meaning they never reached those pages at all.

Key Considerations for Digital Marketing and SEO

While the mantra that "Google renders JavaScript" has long held true in traditional web development, the shift in traffic flow from search engines to AI assistants makes this approach risky. To ensure websites remain scannable by AI search engines and LLM-based discovery tools, serving critical content and internal link infrastructure via Server-Side Rendering (SSR) or pure HTML is vital to prevent loss of visibility.

*This news article was prepared based on data published by Search Engine Land.