llms.txt
llms.txt is a proposed standard — a markdown file at /llms.txt that gives LLMs/AI tools a
“curated overview” of a site’s key content, “to provide LLM-friendly content” at inference time. It’s
a concrete, mechanism-level answer to part of the spoke’s GEO
question (how do you actually make a site usable by AI?). Source: llmstxt.org.
The problem it addresses
“Context windows are too small to handle most websites in their entirety,” and “converting complex HTML pages with navigation, ads, and JavaScript into LLM-friendly plain text is both difficult and imprecise.” llms.txt offers a hand-curated, link-and-summary markdown map instead.
vs. robots.txt / sitemaps
A useful contrast for the spoke: robots.txt controls crawler access; sitemaps list all pages; llms.txt is a curated, inference-time overview sized to fit an LLM context. It’s the GEO counterpart to those older files — though, like everything in generative-engine-optimization, its real payoff is unmeasured (adoption by AI vendors is voluntary and uneven) — keep it in the “plausible mechanism, no neutral benchmark yet” bucket the synthesis uses.
Related
generative-engine-optimization · answer-engine-optimization · ai-search-shift · brand-depth-ai-recommendations · customer-success-ai-readable-proof