AnswerDotAI

Free llms.txt Generator: What llms.txt Is and How to Create One Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com. An llms.txt generator builds the /llms.txt file that tells AI agents and chat assistants what your site is and which pages matter. I built a free one that reads your sitemap and writes a spec-style file in a few seconds, with no account: open the free llms.txt generator. It also writes an optional llms-full.txt with a short excerpt of every page. This guide explains what llms.txt is, what it does and does not do for SEO, the exact format from the official proposal, how to use the generator step by step, how to serve the file from Next.js, and how to check that it works. What is llms.txt? llms.txt is a Markdown file at the root of a website (/llms.txt) that gives large language models a short, curated map of the site. It starts with an H1 name and a one-line summary, then lists the important pages as Markdown links with notes. Agents read it to find the right page without parsing the whole site. The format was proposed by Jeremy Howard of Answer.AI on September 3, 2024, and the llms.txt proposal was updated to v2 in August 2026. The proposal lives in an open GitHub repository. Definition: llms.txt is a Markdown index for AI agents, placed at /llms.txt (or at a subpath such as /docs/llms.txt), that summarizes a site and links to its most useful pages. llms-full.txt is a common companion file that includes the content itself, or excerpts of it, in one document. TL;DR - llms.txt is a Markdown file at /llms.txt with an H1, a blockquote summary and H2 sections of links. - Only the H1 is required. An ## Optional section marks links an agent can skip. - It is a guide for agents at answer time, not an access control file like robots.txt. - Google says you do not need it to appear in AI Overviews or AI Mode, so treat it as a low-cost extra for agents and AI assistants, not a ranking trick. - Generate one free at tools.fabwaseem.com, edit it, and serve it as plain text. What is llms.txt used for? llms.txt is used to give AI agents a fast, clean starting point. A coding agent can read a library's llms.txt to find the right API page. A chat assistant browsing your site can read it to learn who you are and where your pricing, docs or blog live. It saves tokens because the agent skips navigation, ads and scripts. The proposal says the files are used most heavily for software documentation, and that the AI labs publish their own: OpenAI (developers.openai.com), Anthropic (docs.anthropic.com) and Google's Gemini API docs (ai.google.dev). It also lists platforms that generate the file automatically, including Mintlify, GitBook, Yoast SEO, AIOSEO and Wix. Typical uses: | Site type | What llms.txt points to | | --- | --- | | Developer docs | Quick start, API reference, SDK guides, changelog | | SaaS product | Product overview, pricing, features, docs, support policy | | Personal or portfolio site | About, projects, CV, contact, best articles | | Blog or publication | Pillar articles, topic hubs, author page | | Store | Category pages, shipping and returns policy, size guides | Does llms.txt help SEO or AI search rankings? llms.txt does not directly improve Google rankings. Google's guidance on AI features and your website says there are no extra requirements for AI Overviews or AI Mode and that you do not need new machine-readable files or AI text files to appear in them. Its value is for agents that choose to read it. So be honest about the trade-off. llms.txt costs a few minutes to publish and can help any assistant or coding agent that fetches it. It will not replace the basics that every AI answer engine still depends on: crawlable pages, a sitemap, clear titles, structured data that matches the page, and content that answers the question in its first lines. If you want to check those basics, run your site through the free AI SEO checker; it also checks for llms.txt. What is the llms.txt format? The format, from the official proposal, is Markdown sections in a fixed order: an optional byte-order mark, an H1 with the site or project name (the only required part), a blockquote summary, optional paragraphs or lists with no headings, then zero or more H2 sections that each contain a list of [name](url) links with optional : notes. Here is the shape: # Site name > One-sentence summary with the key facts an agent needs. Optional notes about how to read the links below. ## Docs - [Quick start](https://example.com/docs/start.md): Install and first request - [API reference](https://example.com/docs/api.md): Every endpoint and parameter ## Optional - [Changelog](https://example.com/changelog) Rules worth remembering: | Element | Required? | Rule | | --- | --- | --- | | H1 (# Name) | Yes | The only required section | | Blockquote (> summary) | No | Short summary with key facts | | Free text | No | Paragraphs or lists, no headings | | H2 file lists | No | Markdown list of [name](url): notes | | ## Optional | No | By convention, links an agent can skip | | Location | Yes | /llms.txt at the root or any subpath | The proposal also suggests offering a clean Markdown version of important pages at the same URL with .md added, and linking them with rel="alternate" type="text/markdown". That is optional, and many sites start with the index alone. What is the difference between llms.txt, llms-full.txt, robots.txt and sitemap.xml? robots.txt says which crawlers may fetch which paths. sitemap.xml lists every indexable URL for search engines. llms.txt is a short, curated Markdown guide for AI agents with notes about each link. llms-full.txt is a larger companion that holds page content or excerpts in one file, so an agent can read everything in a single request. | File | Purpose | Audience | Format | Size | | --- | --- | --- | --- | --- | | robots.txt | Allow or block crawling (RFC 9309) | All crawlers | Plain text directives | Tiny | | sitemap.xml | List every indexable URL (sitemaps.org) | Search engines | XML | Can be large | | llms.txt | Curated map with notes | AI agents and assistants | Markdown | Small, fits in context | | llms-full.txt | Content or excerpts in one file | AI agents | Markdown | Larger | The proposal makes the same point: a sitemap often lacks LLM-readable versions, leaves out helpful external links, and is usually too big to fit in a model's context window. llms.txt is the short version. How do I create an llms.txt file with the free generator? Enter your homepage URL in the free llms.txt generator. It reads Sitemap: lines in your robots.txt, then your sitemap (falling back to homepage links), visits up to 30 pages, groups them into sections and writes llms.txt plus llms-full.txt. Review the preview, edit it, then copy or download it. Step by step: 1. Open the tool at tools.fabwaseem.com. 2. Pick a mode. Use Site URL to crawl from your homepage, or Paste URLs to list up to 30 exact pages you want included. 3. Enter your homepage, for example https://example.com, and run it. The tool reads robots.txt for Sitemap: entries, then the sitemap. If there is no sitemap, it uses links from your homepage. 4. Check the summary. The H1 and blockquote come from your homepage title and meta description. Rewrite the summary so it states who you are and what you offer in one sentence. 5. Review the sections. Pages are grouped by path: the homepage under Main, /blog/ or /articles/ under Blog, /docs/-style API paths under API, /pricing and /features under Product, /about and /careers under About, and privacy, terms and login pages under Optional. Other paths get a section named after their first folder. 6. Edit the notes. Each link carries a note from the page's description. Shorten them to what an agent needs to know. 7. Copy or download llms.txt, and llms-full.txt if you want it. The full file adds a short excerpt (up to about 800 characters) of each page. 8. Publish the file at https://yourdomain.com/llms.txt and confirm it loads as plain text. What the tool does and does not do, so you know the limits: | Behavior | Detail | | --- | --- | | Pages read | Up to 30 pages per run, up to 300 sitemap URLs considered | | Sources | robots.txt Sitemap: lines, sitemap index (up to 4 child sitemaps), homepage links | | Safety | Public http/https only; private, loopback and link-local addresses refused after DNS | | Limits | 8-second timeout per request, size caps per file, a few runs per visitor every 10 minutes | | Output | Editable llms.txt and llms-full.txt, copy and download | | Account | None, free | How do I add llms.txt to a Next.js site? In the Next.js App Router, either put a static llms.txt file in the public/ folder, or create a route handler at app/llms.txt/route.ts that returns the text with a text/plain content type. The route handler is better when your pages come from a CMS, because the file stays in sync with your content. Static option: save the generated file as public/llms.txt. Next.js serves it at /llms.txt. Dynamic option: // app/llms.txt/route.ts import { getPublishedPosts } from "@/lib/posts" export const revalidate = 3600 export async function GET() { const posts = await getPublishedPosts() const lines = [ "# Example Studio", "", "> Example Studio builds Next.js apps and publishes guides on self-hosting and AI search.", "", "## Main", "", "- [Home](https://example.com/): Services and contact", "- [Projects](https://example.com/projects): Case studies", "", "## Blog", "", ...posts.map((p) => `- [${p.title}](https://example.com/blog/${p.slug}): ${p.excerpt}`), ] return new Response(lines.join("\n"), { headers: { "Content-Type": "text/plain; charset=utf-8" }, }) } This site does exactly that: fabwaseem.com is built from published posts and projects. For the rest of the setup (metadata, sitemap and JSON-LD), see my guide to Next.js SEO for AI search. How do I check that my llms.txt works? Open https://yourdomain.com/llms.txt in a browser and confirm it returns HTTP 200 with a text/plain or text/markdown content type, starts with an H1, and has absolute https links that all load. Then give only that URL to an AI assistant and ask it questions about your site to see whether it finds the right pages. A quick checklist: - curl -I https://yourdomain.com/llms.txt returns 200. - The first line is # Your name. - The blockquote is one or two sentences with real facts, not slogans. - Every link is absolute and returns 200, with no redirects to login pages. - The file is small. If it is long, move secondary links to ## Optional. - robots.txt does not block the path for the crawlers you care about. Each AI company documents its own user agents: OpenAI, Anthropic and Perplexity. The AI SEO checker runs the H1 and link checks for you as part of a full report. What should I put in llms.txt? Put the pages that answer the questions people actually ask about you: what you do, pricing or services, docs or guides, key articles, and how to contact you. Use a one-sentence factual summary, short notes per link, and an Optional section for legal pages and archives. Leave out thin pages, tag archives, search pages and anything behind a login. Good and bad examples: | Do | Avoid | | --- | --- | | > Acme is a time-tracking app for agencies, from $9 per user per month. | > The best, most innovative solution for everyone. | | - [Pricing](https://acme.com/pricing): Plans, limits and the free tier | - [Click here](https://acme.com/p?id=7) | | 10 to 40 curated links | Every URL from the sitemap | | Absolute https URLs | Relative links or staging domains | | Legal pages under ## Optional | Legal pages at the top | Common llms.txt mistakes The most common mistakes are serving the file as HTML instead of plain text, missing the H1, using relative links, dumping the entire sitemap, and writing a marketing tagline instead of a factual summary. Each one makes the file harder for an agent to parse or less useful once parsed. - Served as HTML. A framework's 404 page or a rewrite returns HTML. Check the content type. - No H1. The H1 is the one required element. - Headings in the intro. Only H2 headings are allowed, and only for file lists. - Relative links. Agents may read the file out of context. Use full URLs. - Stale links. Regenerate after big content changes, or build it from your CMS. - Expecting a ranking boost. It helps agents that read it. It is not a Google ranking factor. Frequently asked questions Is the llms.txt generator free? Yes. The generator at tools.fabwaseem.com is free, needs no account and adds no watermark or branding to the file. There is a small per-visitor rate limit so the tool stays fast for everyone. Where do I put the llms.txt file? Put it at the root of your domain so it loads at https://yourdomain.com/llms.txt. The proposal also allows subpaths, such as /docs/llms.txt for documentation only, and agents should use the most specific file that covers a page. Does Google use llms.txt? Google's documentation for AI Overviews and AI Mode says you do not need new machine-readable files or AI text files to appear in those features. Treat llms.txt as a help for agents and assistants that read it, not as a Google ranking signal. What is llms-full.txt? llms-full.txt is a companion file that puts page content, or excerpts of it, into one Markdown document so an agent can read everything in one request. The generator writes one with a short excerpt per page. Keep llms.txt as the short index. How often should I update llms.txt? Update it whenever you add or remove important pages. If your site uses a CMS, generate the file from your content with a route handler so it never goes stale. Otherwise, rerun the generator after major releases. Is llms.txt the same as robots.txt? No. robots.txt controls which crawlers may fetch which paths. llms.txt does not grant or block anything. It is a Markdown guide that tells agents what your site contains and which pages matter. Key takeaways - llms.txt is a Markdown map for AI agents at /llms.txt: an H1, a summary and H2 lists of links. - It complements robots.txt and sitemap.xml. It does not replace them and is not a Google ranking factor. - Keep it short and factual, with absolute links and an ## Optional section for secondary pages. - In Next.js, serve it from public/ or a route handler that builds it from your content. - Generate a draft in seconds with the free llms.txt generator, then edit it. Try the llms.txt generator Paste your homepage into the free llms.txt generator, edit the summary, and publish the file today. Then run the page through the AI SEO checker to see what else AI answer engines need from your site. More free tools are on the tools page, and you can see what I build on my projects page. --- Originally published on fabwaseem.com.

つまり、AIに引用してもらうための「自己紹介ファイル」。BtoBは指名検索が命なので、地味だけど効きそう。もう置いてる？👇 AnswerDotAI/llms-txt

🔥 Build Full-Stack Web Apps in Pure Python With FastHTML! #coding #programming #javascript#ai "Tired of wrestling with complex JavaScript frameworks, messy API routes, and endless frontend boilerplate just to ship a simple web app? 🛑 Meet FastHTML—a powerful new open-source Python framework that lets you build lightning-fast, server-rendered full-stack web applications using pure Python! 💻🚀 🔗 Get the tool here: AnswerDotAI/fasthtml 👇 Don't forget to LIKE, SUBSCRIBE, and hit the bell for more cutting-edge open-source tech and developer secrets!" #Python #OpenSource #GitHub #WebDev #FastHTML #TechShorts #ArtificialIntelligence #Developer

Want ColBERT-quality retrieval in your RAG app this afternoon? RAGatouille makes it a few lines of Python, and its top contributor is Benjamin Clavié (@bclavie). Ben (GitHub bclavie (github.com/bclavie)) works on NLP and information retrieval. His site (ben.clavie.eu) says his long-term goal is "figuring out the perfect way to do information retrieval." He does R&D at Mixedbread on multimodal retrieval systems. What he built 1. RAGatouille (AnswerDotAI/RAGatouille): "Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline." 3,968 GitHub stars as of Oct 16. 2. rerankers (AnswerDotAI/rerankers): "A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models." 1,641 GitHub stars as of Oct 16. Both live under the AnswerDotAI GitHub org, and GitHub's contributor list shows him as the top contributor to each. Why late interaction matters A standard dense retriever squeezes a whole passage into one vector. ColBERT-style late interaction keeps a vector per token and matches query and document at a finer grain. RAGatouille packages that research into a few lines of Python, so a solo builder can index documents, search them and fine-tune on their own data. Why rerankers matters A reranker reorders your retrieved results so the most relevant ones reach the LLM first. rerankers puts many reranker and cross-encoder models behind one small API, so you can try a different model by changing a line of config and keep the rest of your pipeline intact. Why this spotlight Ben's libraries carry thousands of stars, and his own GitHub profile shows 212 followers (Oct 16). Because the repos sit under an org, the person behind them is easy to miss. He also shares what he learns in public, from his posts on late interaction to his love for encoder models. That generosity helps the whole retrieval community. How he helps builders - Research made usable: late-interaction retrieval methods with a beginner-friendly API - Low-dependency design: rerankers stays small, so it slots into existing stacks - Open licensing: both projects are Apache-2.0 Try this Beginner: open the RAGatouille README (AnswerDotAI/RAGatouille), index ten of your own documents and run a few queries. Building RAG already? Add a reranking step with rerankers (AnswerDotAI/rerankers) and compare your top-5 results before and after. Who built the retrieval layer you rely on? Reply with a name and we may feature them next. Aurevon Labs · celebrating craft that helps the whole room

all-in-one tool for seo and geo Direct answer An all-in-one SEO and GEO tool wins AI visibility by running classic ranking and answer-engine citation as one workflow: it audits what answer engines can retrieve, tracks whether your pages actually get quoted, fixes the machine-readable context around them, and refreshes them before they rot. One system, two distribution channels. Search handed you a shelf. Answer engines hand you a sentence. In May 2024 Google started placing AI Overviews above the classic results in US search, composing an answer from multiple sources instead of listing ten links to choose between (blog.google). Bing followed with its own generated summaries, and ChatGPT and Claude both added live web retrieval (openai.com, anthropic.com). That changes the shape of the job. The old SERP was a shelf: ten books, and the reader does the synthesis. An answer engine is a research assistant who has already read the books and hands you a paragraph with a few footnotes. If your page was the raw material and someone else's page was the footnote, you paid for the reading and got none of the credit. Two distinct skills fall out of that. SEO gets you into the candidate pool. GEO gets you quoted once the engine is already talking. Most teams staff the first and assume the second happens by gravity. It doesn't. Google's own guidance is blunt that content has to be crawlable, indexable, and actually useful before anything else applies (developers.google.com) — but usefulness is now filtered through a retrieval step nobody can see. What the answer engine actually reads Four things happen between a user's question and your name in the paragraph. Crawl. Your page has to be in the pool. OpenAI documents GPTBot and how to allow or block it (platform.openai.com); Anthropic does the same for ClaudeBot. A surprising number of sites block AI crawlers in robots.txt while their marketing team asks why they never get cited. Check the lock before you blame the key. Retrieve. The engine doesn't hand the model your page — it hands over chunks, ranked by relevance to the query. This is where structure beats word count. Research on long-context models found they attend most reliably to the beginning and end of a long input and least to the middle (huggingface.co). If your answer sits 600 words into a warm-up, you've hidden it in the least-read part of the document. Ground. Structured data gives the model entities, authorship, and dates it can attach to a claim. schema.org's vocabulary defines the types (schema.org), and Google documents how structured data earns rich treatment in its own surfaces (developers.google.com). Agents that connect to tools through the Model Context Protocol read the same kind of machine-readable signals (modelcontextprotocol.io). Dates matter more than most people think: an undated claim competes badly against a dated one. Synthesize. The model writes a sentence and picks two to six sources. You are either in that set or you are background noise. This is the part a person can do for ten pages and cannot do for two thousand. An audit that tells you, per URL, whether the page is retrievable and quotable is the difference between a project and a habit. Worked example: the page that started getting quoted A real shape we see often. A B2B company has a guide called "How to size a warehouse lease." It ranks okay. It has never been cited by an answer engine. Before: 900 words of context. The actual answer — the sizing formula — appears in the fourth H2. Title is "Warehouse Leasing: Everything You Need to Know." No author, no date. Product names appear as four different strings across the page. After: Three sentences directly under the H1 that answer the question. A table with the variables. An author with credentials. A dateModified that reflects the last real edit. Consistent entity naming. The formula is now the first thing a retrieval step can chunk. None of that is clever. It's the difference between handing someone a book and handing them the index card they asked for. Two supporting pieces finish it. First, an llms.txt file at the root — a plain-text map listing the canonical pages an AI agent should read, per the proposal maintained in the open (AnswerDotAI/llms-txt). Adoption is early and there is no public data showing it changes citation rates, so treat it as a cheap, honest signpost rather than a lever. Second, agent-ready structured data, so the page carries its own entities, author, and freshness instead of making the model guess. The measurement problem, and the tradeoff nobody warns you about You cannot rank-track your way into an answer engine. Responses vary run to run, they're personalized, and they rarely map to a keyword you can put in a column. Position data tells you about the shelf. It says nothing about the sentence. So you sample. You build a fixed set of prompts, run them on a schedule, and log which domains get cited for each. That's citation tracking, and its value is entirely in the prompt set. Here's the trap: if your prompts are your branded queries, you'll score beautifully and learn nothing. Build the set from sales-call questions, support tickets, and the long-tail queries already in your search console. Then hold it fixed, because a prompt set you edit every month measures your edits, not your progress. One honest limit, stated once: there is no public benchmark correlating AI citation share to pipeline. Anyone selling you a dashboard that "proves" GEO ROI is extrapolating. Sample, watch the trend, and treat it as directional evidence alongside your usual demand data. Shipping it in a week Ten pages, not a replatform. 1. Pull your top fifty pages by organic entry and by revenue touched. That's the working set. 2. Run a GEO / AI-visibility audit on those URLs and read the failures as a list: blocked crawler, slow render, answer buried, no dates, no entities. 3. Rewrite the top of each page. Three sentences that answer the title's question, then the evidence. Keep the depth; move the payoff. 4. Add authorship, dates, and agent-ready structured data so the page can be attributed and trusted without human interpretation. 5. Set a refresh cadence. Answer engines quietly drop pages that stop being current. Decay detection flags the pages whose citations and positions have started sliding, so you refresh the source while you're still in the answer, not after. 6. Publish, then re-sample in thirty days. Same prompts, same questions, diff the cited domains. If your CMS supports IndexNow, wire it up so changes are signalled to participating engines immediately rather than waiting for a recrawl (blogs.bing.com). The teams that do well here aren't the ones with the biggest content budget. They're the ones who treat the answer engine as a reader with a short attention span and a good memory. FAQ Is GEO just SEO with a new name? No, and the difference is mechanical. SEO optimizes for inclusion in a ranked list of links; GEO optimizes for being quoted inside a synthesized paragraph, which depends on chunk-level retrieval, attribution signals, and freshness. They share foundations — crawlability, structure, genuine usefulness — but they are measured differently and often optimized by different edits. Teams that treat GEO as a rebrand of SEO skip the chunking work and wonder why rankings hold while citations don't. Do I need llms.txt to get cited by ChatGPT? No. There is no public evidence that any major answer engine requires it, and there's no public data showing it moves citation rates. It's a low-cost, honest signal that costs one file and a maintenance habit. Add it if you want a clean map for agents; don't build a strategy on it. How do I measure AI visibility without an industry benchmark? Fix a prompt set drawn from real customer questions, run it on a schedule across the major engines, and record which domains get cited for each prompt. Track your citation share over time rather than against an external number. Because results vary run to run, sample enough times that single responses don't move your conclusions. Which schema types matter most for answer engines? Start with the types that establish what a page is and how current it is: Article, FAQPage, HowTo, and Product, plus Organization and Person for attribution. schema.org defines the full vocabulary (schema.org). The goal isn't markup volume — it's removing ambiguity about who said this, when, and about what. Can an all-in-one tool guarantee placement in AI answers? No tool can, and any vendor claiming otherwise is telling you something about their sales process. Answer engines choose sources through retrieval and ranking systems nobody outside can see or control. What a tool can do is make your pages retrievable, well-attributed, current, and measurable, which is most of what's actually in your hands. Next step If you want the failure list for your own top pages — blocked crawlers, buried answers, missing attribution — [Run a free GEO audit of your site](/geo-audit). It gives you the specific URLs to fix first, in the order that moves citations. Related reading - [Agentgateway alternatives for AI visibility](/blog/agentgateway-alternatives-ai-visibility/) - [Conductor AI Search alternatives for AI visibility](/blog/conductor-ai-search-alternatives-ai-visibility/) - [Cloudflare AI Crawl Control alternatives for AI visibility](/blog/cloudflare-ai-crawl-control-alternatives-ai-visibility/)

AI search visibility Direct answer AI search visibility means getting your brand cited inside the answers generated by ChatGPT, Perplexity, and Google AI Overviews. Blue-link rank matters less than three things: being retrievable, quotable, and verifiable. The practical goal is straightforward: publish specific, structured evidence that answer engines can lift with confidence—and then credit. How an answer engine chooses a source Ask an AI search surface a question and it does not begin by ranking ten blue links. It retrieves passages, reranks them against the query, synthesizes an answer, and attaches citations to the sentences it borrowed. Google describes AI Overviews as showing links to supporting sources—the citation is part of the product, not an afterthought (Google AI features). OpenAI's SearchGPT prototype, announced in July 2024, displayed the same behavior: a conversational answer with attributed links (OpenAI). The unit of competition is the passage, not the page. A page can rank fourth and still supply the sentence that wins the answer. That is why two pages on identical topics can have wildly different AI visibility. One contains a clean, self-contained definition. The other buries that definition in the third paragraph of a preamble. If you want citations, write in chunks that survive extraction. Where visibility diverges across ChatGPT, Perplexity, and AI Overviews The three surfaces share retrieval mechanics, yet they reward different signals. Google AI Overviews lean on the Google index, so crawlability, internal linking, and structured data still matter. They also surface in informational, local, and shopping queries—meaning a product page can be cited if it answers a comparison question directly. ChatGPT, especially with search enabled, behaves more like a research assistant. It tends to present a synthesized answer alongside a source list. Perplexity surfaces citations inline, which we find useful for diagnosis: you can see which page supplied each claim. In our audits, the same page often gets cited by one surface and ignored by another for weeks. That is not randomness. It is different retrieval corpora and different reranking priorities. Track each surface separately. A single blended "AI visibility score" hides the decisions you need to make. The citation unit: specific, sourced, structured Answer engines do not need your entire article. They need a quotable unit. The strongest units carry four properties: a direct claim, a named entity, a source or date, and a clear boundary. "Data lineage is the recorded history of data as it moves through systems" is a unit. "In our experience, data lineage is important" is not. Structure gives the machine handles. Schema.org supplies a shared vocabulary for marking up FAQs, HowTo steps, articles, and datasets (schema.org). Google's guidance on creating helpful content still applies: write for people, but make the evidence easy to extract (Google Search Central). A worked pattern: put a one-sentence definition under the H2, follow it with a table of tradeoffs, then add a short "How we know" paragraph with the source. That shape gives the retriever a clean passage and gives the synthesizer a reason to attribute it to you. Vague thought leadership rarely gets cited because there is nothing to quote. A worked example: the glossary page that became a source We once worked with a B2B data company whose "What is data lineage?" page ranked on page three. It opened with three paragraphs about the company's founding story. We rebuilt it around a single question. First, a plain definition. Then a diagram showing source system, transformation, warehouse, and report. Then a table comparing lineage capture methods: parsing SQL, column-level tagging, and runtime observation. Finally, we added a short original observation from their support tickets—the three questions buyers ask when lineage breaks. We marked the FAQ section with structured data. The page did not suddenly outrank the category leaders. But within a few weeks it began appearing as a cited source in AI Overviews for long-tail queries like "column-level lineage vs runtime lineage." No magic was involved. The page had become the easiest passage to lift for that specific comparison. If a page lacks a named method, a clear boundary, and a date, the model must paraphrase without credit—or skip it. Measure citations, then defend them from decay Rank tracking alone will not tell you whether AI search visibility is improving. You need a query set that mirrors real buyer questions and a log of which URLs get cited on each surface. Manual checks in a clean browser work for a small set. At scale, answer-engine citation tracking turns that log into a dashboard: query, surface, cited URL, date, and snippet. The second job is maintenance. Citations decay. A source cited in January may vanish by April because a competitor published a fresher table, or because the page's statistics went stale. Content decay detection and refresh flags pages whose citation frequency drops, then prompts a targeted edit: new date, new example, sharper definition. This is unglamorous work. It is also the difference between a one-time spike and a durable presence. No public benchmark yet tells you the average citation lifespan, so treat your own log as the ground truth. Make the site readable to machines and agents Two technical layers are worth building now. The first is a curated map for language models. An llms.txt file is a proposed convention that gives AI crawlers a clean list of your most important pages and what they contain; llms.txt generation can turn your sitemap and content inventory into that index (llms-txt on GitHub). The second is agent-ready structured data. As AI agents begin to fetch live data rather than only read static pages, they will look for machine-readable facts about your products, policies, and availability. Schema.org markup is the baseline. The Model Context Protocol, introduced by Anthropic, is one emerging way for agents to connect to external tools and data sources (Model Context Protocol). You do not need to rebuild your site around agents today. You do need to stop hiding facts inside images and PDFs. A crawl-and-extract review will show you which pages are already extractable and which remain invisible to retrieval. FAQ Does AI search visibility replace SEO? No. It extends SEO. Crawlability, internal linking, and page speed still decide whether an answer engine can retrieve your content. The unit of optimization changes: you are shaping passages for extraction and attribution, not merely pages for ranking. SEO gets you into the corpus. AI visibility gets you into the sentence. How long does it take to earn an AI citation? There is no reliable public timeline, and anyone who promises one is guessing. In practice, fresh pages can be cited within days for low-competition queries. Competitive terms take longer because the retrieval corpus must recrawl and rerank. The bigger variable is whether your passage is the cleanest answer available. We have seen a single well-structured table change citation behavior faster than a full site redesign. Do llms.txt files guarantee citations? No. An llms.txt file is a convenience for crawlers and agents, not a ranking factor you control. It can help a model find your best pages. It cannot force a citation. Treat it like a robots.txt for content discovery: useful hygiene, not a lever. Citations still go to the pages that answer the question best. Can I track citations in ChatGPT, Perplexity, and AI Overviews? Yes—but decide how much manual work you want. You can run a fixed query set in each surface and log the cited URLs. For a larger program, answer-engine citation tracking automates that log and shows which pages gain or lose citations over time. The key is to track surfacing and attribution separately, because a brand mention without a link is not the same as a citation. What content types get cited most? No public benchmark yet ranks content types by AI citation rate. From our own audits, the reliable patterns are definitions, comparison tables, step-by-step instructions, original data, and clear FAQ answers. Pages that combine two or three of those patterns tend to outperform essays. The common thread is extractability: the model can pull one clean unit and know exactly what it means. Next step If you want to see where your site stands, start with the queries that matter to your buyers. [Run a free GEO audit of your site](/geo-audit). Related reading - [answer engine optimization software](/blog/answer-engine-optimization-software-ai-visibility/) - [best AI visibility tools](/blog/best-ai-visibility-tools-ai-visibility/) - [best generative engine optimization platform](/blog/best-generative-engine-optimization-platform-ai-visibility/)

best generative engine optimization platform Direct answer The best generative engine optimization platform is the one that matches your bottleneck: a visibility auditor if you're invisible, a citation tracker if you're already mentioned, or a content-refresh system if your pages are decaying. For most teams, start with an audit. Then tie every fix to a measurable answer-engine outcome. Why it matters Answer engines no longer just rank pages. They synthesize an answer and attach a short list of citations. Google's AI features and Bing's Copilot both pull from indexed sources and cite them (blog.google, blogs.bing.com). If your page is not in that synthesized answer, you don't lose a position — you lose the question. The click may never happen. The competitive surface is moving just as fast. ChatGPT search and similar tools answer directly, and OpenAI documents crawlers like GPTBot that decide what can be read (openai.com, openai.com). GEO, then, is part editorial, part technical, part measurement. The platform you pick should map to those three jobs. It should not merely promise "AI visibility." How answer engines choose what to cite Mechanism: retrieval first. The engine chunks and embeds pages, then retrieves passages that answer the query. It prefers passages with a clear claim, a named entity, and a reason to trust the source. Structured data helps because it removes ambiguity about what the page is and who wrote it (schema.org, developers.google.com). Then comes grounding. The model checks the retrieved passage against the query and against other sources. If your paragraph says "we are the best" but never defines the problem, the engine has nothing to quote. If your page answers a sub-question in the first two sentences, it gives the engine a clean unit to cite. Freshness is the third lever. Google's helpful content guidance pushes people-first pages that demonstrate experience (developers.google.com). In practice, that means updating examples, dates, and product comparisons. A stale page can still rank in classic search while disappearing from answer engines because the model prefers recent corroboration. The comparison table: platforms that show up in answer-engine results When we ask answer engines for a GEO platform, a handful of names recur. This is a field observation, not a ranked benchmark; no public data ranks these platforms head-to-head on citation lift. Use the table to shortlist by workflow, then run your own test with your own queries. | Platform / domain | What it tends to surface | Where it fits | Watch-outs | |---|---|---|---| | Otterly.AI / Otterly.ai | AI search visibility, prompt-level brand tracking | Teams tracking share of voice across answer engines | Needs a stable prompt set; otherwise the signal wobbles | | airops.com | Content operations and AI-assisted production | Teams scaling optimization work across many pages | Requires editorial guardrails so output stays specific | | athenahq.ai | Answer engine analytics and competitive gaps | SEO/GEO leads comparing share against rivals | Data hygiene matters; clean domains and prompts first | | goodie.ai / goodieai.com | AI visibility dashboard | Teams starting GEO and needing a single view | Crawl access limits what any tool can see | The honest take: no platform replaces the work of making a page worth citing. Tools are telescopes. They show you where the answer engine is looking. They don't write the sentence that gets quoted. A worked example: three weeks from invisible to cited We ran this play with a B2B SaaS client selling ledger reconciliation. They ranked page two for "automated reconciliation," but answer engines cited three competitors and never mentioned them. Week one: a GEO / AI-visibility audit showed the gap. The query set was not "reconciliation software" — it was "how to reconcile Stripe payouts with NetSuite," "best way to close the books faster," and "what to do when transactions don't match." Week two: we rewrote the page around those sub-questions. Each section opened with a direct answer. We added schema for SoftwareApplication and FAQPage, and linked the claims to primary docs (schema.org, developers.google.com). We also generated an llms.txt file so crawlers had a clean map of the site's canonical answers (AnswerDotAI/llms-txt). That is llms.txt generation, but the outcome matters more than the file: the crawler stops guessing which page matters. Week three: answer-engine citation tracking showed the first citation on a long-tail query. Not a flood. One citation. But it was on the exact question their sales team heard on calls. From there, content decay detection & refresh flagged two older posts that had lost citations after a product rename. We updated the examples and re-earned the mentions. The lesson: GEO is a loop, not a launch. For agent-driven traffic, agent-ready structured data is how you make the page legible to MCP-style tools (modelcontextprotocol.io). It is plumbing, but plumbing is what lets an agent quote you without a human in the loop. What to measure (and what not to) Measure citation share, not raw mentions. A page can be mentioned in a listicle and still never be the source the answer engine quotes. Track the queries where you appear as a cited source, the queries where competitors appear, and the queries where the answer engine says "no public data" or hedges. Those hedges are openings. Also measure the downstream: branded search, demo requests that say "I saw you in ChatGPT," and sales-call mentions. Attribution is messy. No public data gives a clean multiplier from AI citation to pipeline. So treat citation share as a leading indicator, not a closed-loop ROI number. Avoid optimizing for "number of AI mentions" alone. It rewards volume over precision. One citation in a high-intent answer beats ten mentions in generic roundups. FAQ What is the best generative engine optimization platform? The best platform is the one that fits your current bottleneck. If you don't know where you stand, start with an audit. If you already get mentions, prioritize tracking and refresh. If your content is stale, prioritize decay detection. No single tool wins for every team because the jobs differ. How is GEO different from SEO? SEO optimizes for ranked links; GEO optimizes for synthesized answers and citations. Google's AI features and Bing's Copilot both cite sources inside an answer (blog.google, blogs.bing.com). That means your page must be quotable, not just clickable. The overlap is real, but the unit of success shifts from position to citation. Do I need structured data for GEO? Yes, structured data helps answer engines identify entities, authors, and page types. Schema.org provides the vocabulary, and Google documents how to implement it (schema.org, developers.google.com). It won't guarantee a citation. It reduces ambiguity, which is often the difference between being retrieved and being ignored. How long does GEO take? There is no public data that gives a reliable timeline. Crawl frequency, site authority, and query competition all move the date. In our experience, early signals can appear in weeks, but durable citation share takes months of iteration. Track weekly and expect noise. Can I do GEO without a platform? You can. Spreadsheets, manual prompts, and a content calendar will get you started. A platform becomes useful when you need to monitor many queries, catch decay, and prove which fixes earned citations. The tool is not the strategy; it is the memory and the scoreboard. Next step If you want to see where answer engines cite you today — and where they cite everyone else — start with a free GEO audit. We'll show you the queries you're missing, the pages that need refresh, and the structured data that makes you legible to agents. [Run a free GEO audit of your site](/geo-audit) Related reading - [answer engine optimization software](/blog/answer-engine-optimization-software-ai-visibility/) - [best AI visibility tools](/blog/best-ai-visibility-tools-ai-visibility/) - [GEO tools](/blog/geo-tools-ai-visibility/)

MCP server Direct answer An MCP server wins AI visibility the moment an answer engine needs a fact and reaches for yours. You stop hoping a crawler re-reads your page, and start publishing typed tools, clean resources, and citable outputs that models can invoke, quote, and link inside a generated answer. The shift: from crawling to calling For twenty years, visibility meant ranking. You tuned a page, earned links, waited for a crawler to come around again. Answer engines changed what competes. ChatGPT, Perplexity, and Google's AI Overviews assemble a response out of retrieved passages and — increasingly — out of tool calls. Google has said its AI Overviews reach a large audience blog.google. The mechanism underneath is different: the model doesn't rank your page, it decides whether to call a tool, quote a passage, or skip you completely. MCP is the protocol that makes tool calls portable. Anthropic introduced it in November 2024 anthropic.com. OpenAI added MCP support to its Agents SDK in March 2025 openai.com. Why that matters: an MCP server is not a website. It's a menu of functions an agent can run. Ask "What's the current mortgage rate for a 30-year fixed?" and the agent can call a rates tool, pull structured data, and cite the source. If your competitor runs that tool and you don't, the answer is theirs. The tradeoff is real. An MCP server only helps inside clients that speak MCP. ChatGPT's consumer app won't let you point at any server you like. Perplexity doesn't expose a general MCP client today. So an MCP server is a forward bet, not a switch you flip for instant citations. But the agents are coming, and they will call whichever tools are easiest to call. What an MCP server actually exposes MCP servers expose three primitives: tools, resources, and prompts modelcontextprotocol.io. Tools are functions with typed parameters. Resources are readable data — a file, a database row. Prompts are templates the client can use. For visibility, tools matter most, because they let the model act. Design your tools the way you'd brief a brilliant intern who has never seen your product. The name should match the question: get_shipping_estimate beats shipping_v2. The description should say what the tool returns and when to reach for it. The output should be small, structured, citable. A JSON blob with a source_url field and a last_updated timestamp does more for citation than a 4,000-word marketing page. Schema.org plays a parallel role. It tells crawlers what a page means schema.org. MCP tells agents what a service can do. You need both. We generate llms.txt files so agents can discover your public content without guessing which pages matter AnswerDotAI/llms-txt. That file is a map, not a ranking factor — but it cuts the odds an agent misses the one page that answers the question. The four moves that make an MCP server citable 1. Write tool descriptions for the query, not the API. The model sees a list of tools. If your description reads "Returns a list of objects," the model has no idea when to call it. If it reads "Returns current 30-year fixed mortgage rates by state, with source URL and timestamp," the model knows. Worked example: a fintech client renamed getRates to get_mortgage_rates_by_state. The model matched more queries. The mechanism is simple: the model matches words. 2. Return a citation payload. Every tool result should carry source_url, source_title, and retrieved_at. Now the model can quote you and link back. Return a bare number, and the model has nothing to cite. It answers without attribution — or attributes to a competitor. 3. Keep latency low and failures graceful. Agents run on timeouts. A tool that takes 9 seconds may be abandoned mid-call. A tool that returns a 500 error teaches the model to avoid you. Cache where you can, and return a short error with a fallback URL. 4. Version and sign your server. Agents need to trust the tool. A signed manifest with a version number and a changelog removes ambiguity. Change a field name, version it. Broken tools get dropped. We track answer-engine citations so you can see whether your MCP responses actually become sources in generated answers. That's the feedback loop: ship a tool, watch the citations, refine the description. Where ChatGPT and Perplexity actually stand ChatGPT's consumer product is not an open MCP client. You can't paste a server URL and expect it to be used. But OpenAI's Agents SDK supports MCP, which means developers building on OpenAI can wire your server into their agents openai.com. That's a distribution channel: every agent built on that SDK is a potential caller. Perplexity has built its own retrieval stack. It does not document general MCP support. So if your strategy is "Perplexity will call my MCP server tomorrow," you're guessing. The better bet is to make your public pages answer-ready and your MCP server agent-ready. Both feed the same answer engine. Google's AI Overviews and Gemini use Google's own retrieval and structured data. Google has published guidance on AI features and structured data developers.google.com. The same facts that make a page eligible for a rich result make it easier for an MCP tool to return. We add agent-ready structured data to your pages so crawlers and MCP tools agree on the facts. When your page says one price and your tool says another, the model hesitates. A worked example: the API docs company Picture a company that sells a payments API. Their docs are good, but ChatGPT keeps citing a competitor. The fix isn't another landing page. It's an MCP server with three tools: - search_docs(query) returns the top three doc snippets with canonical URLs. - get_code_example(language, endpoint) returns a runnable snippet and the doc URL. - get_pricing(plan) returns current pricing with a source URL. When a developer asks an agent "How do I handle idempotency in Payments API X?", the agent calls search_docs, gets a snippet, and cites the doc. The citation isn't guaranteed — the model still chooses. But the tool removes friction. The competitor's docs are only a web page. Yours are a function. The tradeoff: building and maintaining an MCP server costs engineering time. You handle auth, rate limits, schema changes. But the cost is bounded. A single well-described tool can cover hundreds of queries. We run a GEO / AI-visibility audit that checks whether your MCP tools are discoverable and citable, so you know where to start. The measurement trap Don't measure MCP success by rankings. There are no rankings in an answer engine. Measure citation share: how often your domain appears as a source in AI answers for your target questions. Measure tool call share: how often your MCP server is invoked compared to competitors. Measure answer accuracy: does the model quote your price correctly? There is no public benchmark for how often MCP servers get cited in ChatGPT or Perplexity answers. That's not a reason to avoid the channel. It's a reason to instrument it yourself. We detect content decay in the pages your MCP server cites and flag them for refresh. Stale pages produce stale tool outputs, and stale outputs lose citations. FAQ Q: Do I need an MCP server to rank in AI answer engines? No. Answer engines still retrieve from the open web. An MCP server is an additional path for agents that support tool calls. If your pages are well-structured and citable, you can win citations without MCP. But if competitors expose tools and you don't, you're invisible in that path. Q: Does ChatGPT use MCP servers? ChatGPT's consumer app does not let users connect arbitrary MCP servers today. OpenAI's Agents SDK supports MCP, so developers building agents on OpenAI can use them openai.com. The practical effect: your server can be called by third-party agents, not by every ChatGPT user. Q: What's the difference between an MCP server and an API? An API is built for developers who know your docs. An MCP server is built for models that don't. The difference is description and output shape. A REST API returns JSON; an MCP tool returns JSON plus a description that tells the model when to call it. That description is the visibility layer. Q: How do I make my MCP server discoverable? Register it in MCP server directories, document it in your llms.txt file, and link to it from your developer docs. Use clear tool names and descriptions. Discovery follows the same pattern as any developer tool: make it easy to find and easy to try. Q: What should I measure first? Start with citation tracking for your top 20 questions. Then add tool call logs if your server is public. Compare before and after. If citations rise, double down on the tools that get called. If they don't, rewrite the descriptions before you rewrite the server. Next step If you want to see how your site currently appears in AI answers — and where an MCP server would change the picture — start with the raw data. [Run a free GEO audit of your site](/geo-audit). Related reading - [Agentgateway alternatives for AI visibility](/blog/agentgateway-alternatives-ai-visibility/) - [Ahrefs Brand Radar alternatives for AI visibility](/blog/ahrefs-brand-radar-alternatives-ai-visibility/) - [AI Visibility Index](/blog/ai-visibility-index/)

For anyone interested in an ergonomic API over this-- I ported sqlite-utils to apsw a while back: AnswerDotAI/apswutils I've been using it a lot over the last year or two and it's worked great.