Probably not, and you can settle it in two minutes by reading one file on your own site. Cloudflare did set a default block against AI crawlers, and if your site sits behind Cloudflare, some of that may be switched on without you ever clicking it. What the default blocks is training, not the crawler tied to ChatGPT’s search results.
Why people started asking this
On July 1, 2025, Cloudflare put out a press release headlined “Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model”. The line everyone repeated is this one: “Upon sign-up with Cloudflare, every new domain will now be asked if they want to allow AI crawlers, giving customers the choice upfront to explicitly allow or deny AI crawlers access.”
Read it twice. It describes new domains, at sign-up. The release does not say every site already on Cloudflare was flipped to blocked that day.
It only touches you at all if you are behind Cloudflare. W3Techs, whose reports are updated daily, put Cloudflare at 24.8% of all websites in its August 2026 report. Plenty of trades sites are on it and the owner has no idea, since a web company set it up years ago and moved on. Whether a local business needs a CDN at all is a fair question to put to them.
Read your own robots.txt
Type your web address in a browser and add /robots.txt to the end. So yoursite.com/robots.txt. Press enter.
You will get plain text, or an error. An error or a blank page usually means there is no file at all, so nothing on your site is asking a crawler to stay away.
If there is text, you are looking for two lines that sit together:
- A line starting “User-agent:” followed by a name.
- A line under it reading “Disallow: /”.
Those two together ask that named crawler to stay off the whole site. A “Disallow:” with nothing after it, or one pointing at a folder like /wp-admin/, is not a sitewide block.
The names to look for, and the one that is missing
Cloudflare’s developer page for managed robots.txt, last updated August 3, 2026, shows the file it writes on your behalf. The names it disallows are Amazonbot, Applebot-Extended, Bytespider, CCBot, ClaudeBot, Google-Extended, GPTBot and meta-externalagent. Eight names. Go and count them yourself.
Now hold that against what OpenAI publishes. OpenAI’s own crawler page lists four programs it runs: GPTBot, OAI-SearchBot, ChatGPT-User and OAI-AdsBot. That page carries no update date.
GPTBot is on Cloudflare’s list. OAI-SearchBot is not. Neither is ChatGPT-User.
GPTBot is the one OpenAI ties to training. OAI-SearchBot is the one tied to search, and OpenAI’s page says: “To help ensure your site appears in search results, we recommend allowing OAI-SearchBot in your site’s robots.txt file and allowing requests from our published IP ranges below.”
So the fear that Cloudflare cut you out of ChatGPT does not match Cloudflare’s own file. The default aims at training. The search crawler is left alone.
That is what the file says, and no more. It is not evidence anything read you. Whether a crawler actually turned up is a separate check you run against your own access logs, and what ChatGPT reads on your website covers the trap where a page loads fine for you and is still shut out.
What changes on September 15, 2026
Cloudflare posted an update on July 1, 2026, titled “Your site, your rules: new AI traffic options for all customers”. It sorts AI visitors into three groups. Search “collects or indexes your content, so it can answer questions about it later”. Agent is “acting, usually in real time, on a person’s behalf”. Training is “a crawler taking your content to train or fine-tune a model”.
Then the sentence to read slowly: “On September 15, 2026, we’ll be setting new defaults for each of these three classifications. For all new domains onboarding to Cloudflare, the categories of Training and Agent will be blocked by default on the pages that display ads, while Search will remain allowed by default.”
Three limits sit inside that. New domains, so not yours if you are already on. Only pages that display ads, which a contractor site almost never does. And Search stays allowed.
Cloudflare adds that an owner who wants something different “can easily mark this in their Security settings any time leading up to September 15”.
What a robots.txt file cannot do
Cloudflare says it flat on its own documentation page: “robots.txt compliance is voluntary. The file expresses your preferences, but it does not prevent crawlers from accessing your content at a technical level. Some crawler operators may disregard your robots.txt directives (instructions like Disallow: /) and crawl your content regardless.”
That cuts both ways. A clean file is not proof you are being read. A blocking line is not proof nothing is reading you.
Here is the honest part. If you are not on Cloudflare, or your file has no sitewide Disallow sitting under an AI name, there is nothing here to fix and nothing here to buy. Close the tab. This is a five minute check, not a service.
One more gap in the evidence. Cloudflare has a separate control that turns crawlers away at the door with a 403 or 402 response, described on its page for managing AI crawlers, last updated July 28, 2026. That kind of block never shows up in robots.txt. If you suspect it, the only way to know is to ask whoever holds the Cloudflare login.
What to do next
- Open yoursite.com/robots.txt and read the whole thing. It is usually shorter than a receipt.
- Search that page for GPTBot and for OAI-SearchBot. Note whether a “Disallow: /” sits under either.
- Decide about GPTBot on purpose. Blocking it says do not train on my words. Allowing it does not put you in an answer.
- Leave OAI-SearchBot alone unless you have a reason to change it. That is the one tied to search results.
- Find out who holds your Cloudflare account, if there is one. Ask them to open Security settings and show you the AI crawler choice that was made at sign-up.
- If the file was written by a plugin or a host you stopped using, sort that out first, since a plugin nobody maintains leaves its code behind when you switch it off. A stale rule outlives the person who wrote it.
None of this puts your business into an AI answer. It only clears away one reason you would be left out. The slower work sits after that: pages a machine can read, and facts that agree with each other across the site and the listing. That is the search checks side of the job, and it comes with no promise of a mention.
If you would rather someone else looked, the free scan asks AI assistants the questions your customers ask and writes down which businesses get named.