How to Create an llms.txt File for Your Website

Creating an llms.txt file should be a short editorial workflow, not a documentation project that never ships. The goal is to publish a small, trustworthy map of the pages an AI system should understand first.

Start with a focused source list

Choose one input that gives you a reliable starting point: an XML sitemap, your homepage, or a manual list of URLs. A sitemap works well for broad coverage. A manual list is better when you want to launch with a smaller, high-confidence set of product, documentation, and support pages.

Avoid treating every URL as equally valuable. Filter out account pages, search results, duplicate archives, and temporary campaigns unless they explain something a user or AI assistant genuinely needs.

Generate a draft, then edit it

Use the free llms.txt generator to turn your source list into a categorized Markdown draft. It gives you a fast first pass, but the final file should still reflect your judgment.

Keep headings practical. Product, Documentation, Integrations, Support, and Company are easier to scan than vague groups. Write descriptions that say what a page contains and who should read it.

The best description is specific enough to guide a retrieval decision and short enough to scan in a list.

Check the file before publishing

Run the draft through the llms.txt validator. Look for a single clear H1, meaningful H2 sections, valid Markdown links, readable descriptions, and pages that still resolve correctly.

A quick review checklist:

1. Does every listed page deserve to be in the file?
2. Can a reader understand each link without opening it?
3. Are the sections named by user intent rather than internal teams?
4. Does the file avoid outdated URLs and duplicate content?

Publish it at the root of your site

Download the final file and serve it at /llms.txt. Test the final URL in a browser, then add the file to your regular content-review process. When your site launches a new product, docs area, or support flow, update the index with the same care you would give your navigation.

For workflows that need page content in addition to navigation, pair the index with the llms-full.txt generator. The smaller index tells an AI where to look; the full file provides selected readable source material in one place.