Google Says You Don’t Need llms.txt for AI Search Visibility – But Chrome Lighthouse Is Checking for It Anyway

The llms.txt contradiction that every SEO, CMO, and developer needs to understand: Google Search Central says it’s not needed for AI Overviews, but Chrome’s Agentic Browsing audits flag its absence. Here’s the full breakdown – including what John Mueller, Lily Ray, and the LinkedIn SEO community are saying.

The Contradiction That Broke SEO Twitter

On May 15, 2026, Google Search Central published its first official guide on optimizing for generative AI features. The message was clear:

“You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in generative AI search.”

Less than a week later, Chris Long (Co-founder at Nextiv) discovered something buried in Chrome’s new Agentic Browsing audits documented at developer.chrome.com that directly contradicts this stance:

“llms.txt: Checks for the presence of a machine-readable summary at the domain root.”

This is not a typo. This is not a draft. This is a live Chrome Lighthouse audit that checks whether your site has an llms.txt file, the exact file Google Search Central told you not to worry about.

THE CORE QUESTION: If llms.txt is not needed for AI search visibility, why is Google’s own browser infrastructure team checking for it? The answer reveals a much bigger shift than most SEOs realize.

The December 2025 origin story

“This contradiction has roots. In December 2025, Google’s own CMS platform began auto-deploying llms.txt files across Google developer properties, including developer.chrome.com and web.dev. The Search Central team removed the file from their docs within hours. Other teams didn’t notice and left it up. This pattern one Google team deploying what another Google team rejects is the same dynamic playing out now between Search Central and Chrome Lighthouse.”

What Is Chrome’s Agentic Browsing Audit?

Chrome’s Agentic Browsing audits are a new set of Lighthouse checks designed to evaluate whether your site is structured for AI agents and machine interaction. These audits go far beyond traditional SEO metrics. They assess:

• Visibility: Confirming that content is not hidden from the accessibility tree while being interactive
• Cumulative Layout Shift (CLS): Measuring visual stability, which is critical for agents relying on element positioning
• Accessibility tree construction: How well AI agents can parse your DOM structure
• txt presence: Whether a machine-readable summary exists at the domain root

This is not a search ranking signal. It is an agent-readiness signal. And that distinction is everything.

Lighthouse documentation describes llms.txt as an ’emerging convention’ at llmstxt.org, a developer-driven standard, not a Google initiative.”

John Mueller’s Response: The ‘Discovery vs. Functionality’ Distinction

John Mueller, Search Advocate at Google.com and leader of the Search Relations team, responded on social media to clarify the apparent contradiction. His explanation introduces a framework that every CMO needs to understand.

Mueller’s short answer was direct:

“The short answer is that it’s not done for search. There’s more to websites than just SEO :-).”

He then expanded with a nuanced distinction:

“The longer & nuanced version is that it’s worth separating “discovery” (finding the website or pages with a global search engine) vs “functionality” (once someone has found the page, helping them to best do the task they want to do).”

He compared it to CTAs on traditional pages: you don’t add CTAs for SEO (to be found), but if you’re responsible for the website overall, ensuring a high ‘discovery rate’ (SEO) together with a high conversion rate is useful to justify your work.

On the developer documentation context specifically, Mueller explained:

“AI coding has gotten very popular, and these coding systems can be efficient and accurate with the code they produce if they can easily read / parse reference material, such as developer documentation… it can help to give them a way to understand the context of the documentation they’re looking at, as well as a simplified version of the reference page (eg, in markdown). OF COURSE they can read HTML just fine, so this is imo more of a temporary crutch, perhaps to save some tokens.”

And the critical line for non-developer sites:

“For non-developer sites, I don’t think this makes much sense, even with more agentic traffic in the future (and if you check your logs, you’re not getting a lot of that at the moment). Making a markdown version of a shoe’s specs is not going to get you more sales (competitors appreciate it tho).”

Mueller’s final word – the line that should be pinned to every SEO’s dashboard:

“And (I know, nobody reads this far), if you think this is important to prepare for when agents are everywhere: your site (all sites) have much more important things to do for SEO than to prepare for a potential future situation that may or may not come. Prioritize needs before dreams.”

What This Actually Means: The ‘Discovery vs. Functionality’ Framework

John Mueller has given us a clean mental model to resolve the apparent contradiction. Here it is in plain language:

This is not Google being inconsistent. This is two different Google teams optimizing for two different use cases and both are correct within their own context.

The Bigger Picture: Technical SEO Is Expanding Into Agentic Readiness

This feels less like ‘SEO is dead’ and more like technical SEO expanding into a new frontier. Here’s what the signals point to:

• Google is increasingly thinking beyond traditional crawling and indexing. The Agentic Browsing audits are not about search rankings they are about whether AI agents can successfully interact with your site.
• Accessibility, semantic structure, layout stability, and machine readability are becoming more important. These were always good practices. Now they have an additional audience: autonomous agents.
• Agentic AI systems may favor sites that are easier to parse, summarize, and navigate efficiently. This is not a ranking factor today. There may not be one tomorrow. But the infrastructure is being built.
• Google still says llms.txt is not a ranking factor, but its own tooling is now checking for it. This is the ‘functionality’ layer, not the ‘discovery’ layer.

The question every CMO should be asking: Will AI agents eventually reward sites optimized for machine usability the same way search rewarded sites optimized for crawlability?

What CMOs Should Do Right Now

1. Don’t Panic-Deploy llms.txt

If your vendor is selling llms.txt as an ‘AI search visibility hack,’ push back. Cite Google’s Search Central guide. Ask for named-engine evidence. If they cite Chrome’s agentic audits, ask: ‘Is this for search discovery or agent functionality?’ Those are two different budgets.

2. Audit Your Site for Agentic Accessibility

Run Chrome’s Lighthouse Agentic Browsing audit on your priority pages. Look at:

• Visibility score: Is content hidden from the accessibility tree?
• CLS score: Is your layout stable enough for agents to interact with elements reliably?
• Semantic HTML: Can an agent understand your page structure without visual rendering?
• Form accessibility: Can an agent complete a booking, purchase, or comparison on your site?

3. Prioritize SEO Fundamentals Over Agentic Dreams

John Mueller said it best: ‘Prioritize needs before dreams.’ Your 2026 SEO priorities should be:

• Fix crawlability and indexability issues in Search Console
• Publish non-commodity content with original insights and first-hand experience
• Replace stock imagery with original photography and data visualizations
• Verify Merchant Center feeds and Google Business Profile accuracy
• Build clear topic clusters with strong internal linking

4. Set Up Bing Webmaster Tools AI Performance Reporting

While Google offers no citation data, Bing Webmaster Tools introduced AI Performance reporting in February 2026. It shows total citations, cited pages, grounding queries, and visibility trends over time. This is your first direct signal of AI citation presence. Set it up this quarter.

5. Test, Don’t Guess

As Lou Storiale put it: ‘They all lie. Test, test, test.’ Run controlled experiments. Create two versions of a page – one following Google’s human-first approach, one following Microsoft’s machine-readable approach. Measure branded search growth, direct traffic, and AI citation frequency over 90 days. Let the data decide.

Final Word

The llms.txt contradiction is not a scandal. It is a signal. Google is building infrastructure for a future where AI agents, not just search crawlers – interact with websites. That future is not here yet. But the tooling is being laid down.

For CMOs, the playbook is clear:

• Don’t deploy llms.txt as an SEO tactic. It is not a search visibility lever.
• Do audit your site for agentic accessibility. Lighthouse’s Agentic Browsing checks are free and informative.
• Do prioritize SEO fundamentals. Crawlability, indexability, original content, and technical hygiene still drive 98%+ of your organic traffic.
• Do set up multi-engine measurement. Bing Webmaster Tools AI Performance reporting gives you data Google won’t.
• Do test and measure. The only way to resolve the Google-vs-Microsoft debate for your specific site is to run controlled experiments.

John Mueller’s closing line should be the north star for every SEO program in 2026:

“Prioritize needs before dreams.”

Sources & References

1. Google Search Central – Optimizing your website for generative AI features on Google Search (May 15, 2026)
2. Chrome for Developers – Agentic Browsing audits & llms.txt documentation (developer.chrome.com)
3. John Mueller (@johnmu.com) – Bluesky response to Lily Ray, May 19-20, 2026
4. Lily Ray (@lilyray.nyc) – Bluesky question to John Mueller, May 19, 2026
5. Microsoft Advertising Blog – All in on AI: Discovery to Influence in GEO Part 2 (April 2026)
6. Microsoft Advertising Blog – All in on AI: Discovery to Influence in GEO Part 1 (October 2025)

FAQ: What CMOs Are Asking