llms.txt is a curated source map for the pages a site wants agents and readers to understand first. It is not a crawler access rule and it is not a substitute for XML sitemaps.
Short answer
Use llms.txt for canonical source pages: guides, tools, research, methodology, glossary, and important hubs. Use robots.txt for access control and XML sitemaps for URL discovery.
What to include
- Definition and hub pages.
- Evergreen guides.
- Research and datasets.
- Tool pages.
- Methodology and trust pages.
- Selected case studies.
What to exclude
Exclude duplicate URLs, search results, low-value archives, thin tags, and pages that do not act as sources. The curated file should be small enough to be useful.
How to use this page
Use this page as the operating reference for the topic, then follow the related tools and guides for implementation. The goal is to move from a vague AEO concept to a concrete publishing action: what to check, what to change, and what to measure after the change.
Implementation checklist
- Confirm the target page is crawlable and canonical.
- Write a direct answer near the top of the page.
- Use headings that map to real prompts.
- Add examples, tables, or checklists where the reader needs a decision.
- Link to glossary definitions and deeper guides.
- Track whether answer engines mention the brand, cite the exact URL, or cite a competitor.
Measurement plan
Run a small prompt panel before and after major changes. Record the engine, prompt, cited URL, citation surface, result type, and notes. A page that moves from no mention to domain mention is progress, but the stronger goal is exact URL citation for the claim the page actually supports.
Common misconceptions
AEO is not a single tag, file, or plugin. It is the combination of access, source clarity, structured writing, evidence, internal links, and measurement. A page can have schema and still be ignored if it does not answer a prompt clearly. A page can rank and still fail to be cited if the relevant passage is vague or unsupported.