Content Signals in robots.txt

Content Signals are a short line in robots.txt that states how you want your content used after it is crawled. There are three signals: search results, AI answers and AI training. Each one is yes or no.

RankFlect SEO adds the line for you. It is off by default, so nothing changes in your robots.txt until you turn it on.

Note Content Signals state a preference. They do not block any crawler. To stop a bot from fetching your pages at all, use the AI crawler switches on the same settings tab.

What the three signals mean

Choice in RankFlect SEO	Signal	Default when you turn it on	What it covers
Search (search)	search	Yes	"Show the site in search results with links to it."
AI answers (ai-input)	ai-input	Yes	"Use and quote the content in AI answers, such as AI search and chat assistants."
AI training (ai-train)	ai-train	No	"Use the content to train or fine-tune AI models."

The defaults keep you in search and AI answers and opt you out of AI training. This is the most common choice for sites that want AI search traffic but do not want to feed model training.

The format comes from contentsignals.org.

Turn it on

1. Go to RankFlect SEO → Settings → Robots & AI crawlers.
2. Find the AI usage preferences (Content Signals) card.
3. Turn on Add Content Signals to robots.txt.
4. Set the three switches the way you want. The card shows the exact line that will be added.
5. Click Save changes.
6. Open /robots.txt on your site to check the result.

With the defaults, your robots.txt gets this, right under User-agent: *:

# RankFlect SEO: Content Signals tell AI companies how this site's content may be used.
# search: search results. ai-input: AI answers that quote the site. ai-train: training AI models.
# https://contentsignals.org and https://blog.cloudflare.com/content-signals-policy
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Your AI crawler blocks, extra rules and the sitemap line stay as they were. If the file has no User-agent: * group, RankFlect SEO adds one with Allow: /.

The preview of the generated robots.txt in Tools → File editor shows the same line.

When RankFlect SEO cannot add the line

RankFlect SEO only changes the robots.txt that WordPress and RankFlect SEO generate. It leaves the file alone in these cases:

Situation	What you see	What to do
A real robots.txt file is in your site's root folder	"A real robots.txt file is in your site's root folder" on the card	Add the line by hand, right under User-agent: *, in that file (Tools → File editor, or your host's file manager).
You saved your own robots.txt in Tools → File editor	"You serve your own robots.txt" on the card	Add the line by hand there, or reset the file to the generated one. See File editor.
Your extra rules already contain a Content-Signal: line	Nothing extra is added	Keep your own line, or remove it and use the card.
Settings → Reading → Discourage search engines from indexing this site is ticked	Nothing is added	Untick it when the site goes live.

Every change to these switches is recorded in the Change Log, where you can undo it.

FAQ

Do AI companies follow Content Signals?

Content Signals are a new and voluntary standard. They tell crawler operators what you allow, but no crawler is forced to follow them. Use them together with the AI crawler switches if you want to block a bot outright.

Will this change how Google ranks my site?

No. The search signal only states that you allow search results with links. It does not change how any search engine ranks your pages.

What is the difference between this and blocking GPTBot?

Blocking a bot in robots.txt asks it not to fetch your pages at all. Content Signals let crawlers fetch the page but say what the content may be used for afterwards, for example search and AI answers but not training.

Is this a Pro feature?

No. Content Signals are free in RankFlect SEO.