What Is llms.txt? The File That Makes Your Business Visible to AI

If robots.txt tells search engine crawlers which pages to skip, llms.txt tells AI systems which pages to read first. It is a plain-text file placed at the root of your website that gives AI crawlers a structured summary of your business, your offerings, and your most important content.

The format was proposed by Jeremy Howard in 2024 and has since been adopted by a growing list of sites that want to be accurately represented in AI-generated answers. It is not an official standard — there is no W3C specification — but major AI crawlers read it and incorporate its contents into their understanding of your site.

What Goes in an llms.txt File

A valid llms.txt file includes four components:

• —A title line with your business name (using # Markdown heading)
• —A brief description of what you do (one to three sentences)
• —An ## Optional Details section with additional context
• —A list of key page URLs with short descriptions of each

Here is a minimal example for a plumbing company:

# Houston Plumbing Co. > Emergency and residential plumbing services in Greater Houston, TX. Licensed, insured, 24/7 availability. ## Key Pages - [Services](https://example.com/services): Full list of plumbing services including drain cleaning, water heater install, and leak repair - [Service Area](https://example.com/service-area): Cities served in Harris and Fort Bend County - [Pricing](https://example.com/pricing): Flat-rate pricing guide for common repairs

How AI Systems Use llms.txt

When GPTBot, ClaudeBot, or PerplexityBot crawl your site, they typically visit /llms.txt early in the crawl sequence. The file gives them a high-confidence summary of what your site is about, which pages contain the most important information, and how to describe your business accurately to users.

Without llms.txt, AI systems have to infer your business from whatever pages they happen to crawl. If they hit your blog first, they may categorize you as a content publisher. If they hit a legal disclaimer page, they may miss your core offerings entirely. llms.txt eliminates this ambiguity.

Which AI Systems Read It

As of 2026, the following crawlers check for llms.txt:

• —GPTBot (OpenAI — used for ChatGPT)
• —ClaudeBot (Anthropic — used for Claude)
• —PerplexityBot
• —Google-Extended (used for Gemini and AI Overviews)
• —Cohere-AI
• —You.com

llms.txt vs. robots.txt vs. sitemap.xml

These three files serve different purposes. robots.txt controls access — it tells crawlers which pages they cannot visit. sitemap.xml lists every URL on your site so crawlers can find them all. llms.txt provides meaning — it explains what your site is and what its most important content is. You need all three for complete AI visibility.

How to Create Your llms.txt

You can write it manually in any text editor and upload it to your web root. The file should be accessible at yourdomain.com/llms.txt with no authentication required. Keep it under 2,000 words — AI crawlers process it in a single context window, and longer files risk truncation.