Free
Decide how AI search reads your site
RankFlect SEO gives you the files and switches AI search depends on: an llms.txt index of your key pages, a switch for each AI crawler, and analytics that show which AI bots read which pages.
What it does
llms.txt and AI crawlers
- llms.txt, a clean index of your key pages for AI agents (per language on Polylang sites).
- A robots.txt editor with a switch for each AI crawler, such as GPTBot, ClaudeBot, PerplexityBot and Google-Extended.
- AI crawler analytics: which AI bots read which pages over the last 90 days, split into training, AI search and live user requests.
- XML sitemaps with noindexed content left out, and IndexNow pings for new content.
- The site audit has an AI search category that checks these settings.
How it works
Three steps
- 1
Choose your crawlers
In the robots.txt settings, allow the AI search crawlers you want and block the ones you don’t.
- 2
Publish llms.txt
Turn on llms.txt to give AI tools a clean index of your key pages.
- 3
Watch who reads what
Open AI crawler analytics to see which bots visit and which pages they read.
Free and Pro
Free, with more in Pro
llms.txt, AI crawler switches and AI crawler analytics are free. RankFlect SEO Pro adds Markdown versions of your pages, llms-full.txt and an AI usage policy.
FAQ
llms.txt and AI crawlers: questions
Does llms.txt improve Google rankings?
No. Google has said llms.txt does not affect Google Search. It helps AI agents and tools read your site.
Should I block AI crawlers?
That is your choice. Blocking AI search crawlers can keep your pages out of AI answers, so many sites allow search crawlers and decide separately about training crawlers.
Where does the crawler data come from?
RankFlect SEO records AI bot visits on your own site and keeps 90 days of history.
Read the docs
Keep exploring
Related features
- AI agents and MCPFreeLet Claude, ChatGPT, Cursor or VS Code work on your WordPress SEO through a built-in MCP server, with scoped tokens, approval and undo.Learn more
- Approval, dry run and undoFreePreview every AI agent change with a dry run, hold edits in an approval queue and undo any change, one at a time or as a batch, in RankFlect SEO.Learn more
- SEO AutopilotProScheduled jobs prepare missing meta descriptions, alt text, internal links and audit fixes. Nothing goes live until you approve it.Learn more
Try it on your own site
Free for unlimited sites. Try every Pro feature for 14 days, no card needed.