Cloudflare's 15 September Deadline: Are You About to Disappear from AI Search?

Cloudflare declared its second "Content Independence Day" on 1 July 2026. Behind the marketing wrapper is a change that matters for any UK service SME thinking about AI visibility. New controls have gone live for all customers, including the Free tier. New defaults kick in on 15 September 2026. And roughly one in five websites on the internet sits behind Cloudflare's network, which means a large chunk of UK service SMEs will be affected without necessarily knowing it.

This piece walks through what has changed, why it matters for AI visibility, and the practical checklist. It is aimed at business owners and marketing leads, not developers.

What Cloudflare has actually done

Cloudflare has moved from a single "Block AI Bots" switch to a three-way classification for automated traffic. Every AI-adjacent bot on the internet is now sorted into one of three buckets based on what it does on your site.

Search. Any bot that crawls to build an index it will later use to answer questions. The traditional deal. It takes your content and sends you referral traffic in return. Googlebot's search crawl, Bingbot's search crawl, Perplexity's search crawl, and the AI search behaviours that Google, Bing and others have added to their existing search products all sit here.

Agent. A bot that visits your site in real time on behalf of a human, to get a specific thing done. ChatGPT fetching a page because a user asked a question. A Claude browser session pulling up your pricing page. A Gemini agent buying a train ticket. The key point is that a person is on the other end, waiting.

Training. A crawler that takes your content to train or fine-tune an AI model. Your content becomes part of the model's underlying knowledge. Once absorbed, there is no traffic coming back to you.

Every Cloudflare customer, on every plan tier, can now allow or block each of these three categories independently. That is the operational change.

The change most people will miss: the 15 September defaults

Setting the controls up is one thing. The bigger story is what Cloudflare is doing to the defaults.

From 15 September 2026, on any new domain onboarding to Cloudflare, the defaults change. On pages that display ads, Training bots and Agent bots will be blocked by default. Search bots will remain allowed. The logic is that ad-monetised pages are meant for human eyes, so bots that funnel human attention away, or absorb content into a model without sending anyone back, get blocked by default.

There is a second, larger change that will affect existing Cloudflare customers too. Multi-purpose crawlers (bots that do more than one job, for example Googlebot which crawls both for Search results and to train Google's models) will be evaluated against all their behaviours from 15 September. Not just one of them.

In practice that means: if you have ever ticked "Block AI Training" in your Cloudflare settings, or if you have used the legacy "Block AI Bots" preset, then from 15 September, Googlebot, Applebot and BingBot will be caught in that block because they crawl for training as well as for search. The most restrictive applicable rule wins.

If Googlebot is blocked on your site, you disappear from Google search results. Not "you rank a bit lower". You are not there.

Why this matters for AI Visibility

The visibility problem for UK service SMEs already looked like this: your customers were asking ChatGPT, Perplexity, Google's AI Overviews and Claude the questions they used to ask Google. If those engines could not retrieve your content, you were invisible to that new layer of buying research. The Optimus AI Visibility audit exists precisely because most UK SMEs have not yet checked whether they are visible on those engines, and most are not.

The Cloudflare change adds a new failure mode on top of that. It is now possible for a business to make itself less visible in AI search by accident, through a security setting it configured months ago and forgot about. Or through a default it never opted out of. Two examples.

Example one. A UK insurance broker signed up with Cloudflare in 2024. In mid-2025, worried about AI scraping, they ticked the "Block AI Bots" preset. Job done, they thought. Their site currently ranks in Google. On 15 September 2026, the default logic changes, and Googlebot is now caught in the block because it also crawls for training. Their organic traffic drops off a cliff the following week, and they do not know why.

Example two. A UK accident management company onboards to Cloudflare in October 2026. Nobody touches the AI bot settings. The new defaults kick in silently. Their site displays ads (Google Ads, retargeting pixels, sponsored placements). Training bots are blocked on those pages. Agent bots are blocked on those pages. So when a customer asks ChatGPT "who handles credit hire in Cheshire", the ChatGPT-User agent tries to visit the site to check details, and gets bounced. The business has just been priced out of the agentic layer without knowing the price was even on the table.

Both are avoidable. Both need a five-minute check now, not in September.

How to check whether you are on Cloudflare

You can do this yourself in under a minute.

The easiest way is to go to whoishostingthis.com or check-host.net, paste in your website URL, and look at the result. If Cloudflare shows up as your hosting provider, CDN or nameserver, you are on Cloudflare. Alternatively, if you know who set your website up, ask them. Cloudflare is often bundled into hosting packages by web agencies without the client necessarily knowing it is running.

If you have a Cloudflare login, easier still. Log in, and you will see the domains you manage on the dashboard.

If you find you are behind Cloudflare, keep reading. If you are not, you can stop reading here, but you may want to circulate this piece to any partner or supplier whose site you rely on for referrals.

What to check in your Cloudflare settings

Three things.

First, the current AI bot settings. In the Cloudflare dashboard, go to Security, then Settings, and look for the AI bot management section. Note whether "Block AI Bots" is currently on, and whether the new three-way controls (Search, Agent, Training) have been configured. If someone ticked "Block AI Bots" months ago as a general precaution, that is the setting most likely to bite you on 15 September.

Second, the September default decision. Cloudflare gives every customer the option to opt out of the new default changes before they land. If you want to keep your current behaviour on multi-purpose crawlers like Googlebot (that is, keep them allowed as Search crawlers even if you have Training blocked), you need to opt out in Security settings ahead of 15 September. The opt-out is a single tick box. Do it now while it is on the screen in front of you.

Third, the strategic decision on Agent bots. This is the one worth actual thought. Agent bots (ChatGPT-User, browser-use agents, Claude working through Chrome) visit your site because a human user asked them to. If you sell to consumers or B2B buyers who are increasingly using AI assistants, blocking Agent bots is blocking your future customers from checking you out. For most UK service SMEs, allowing Agent bots is the right call. But it is a decision, not a default.

The Search default is the easiest one. Leave it allowed. That is the traffic that keeps sending you visitors, and blocking it makes no sense for any business that wants to be found.

The bigger point

Cloudflare's change is not the only reason your AI visibility might be quietly deteriorating. It is one of five or six moving parts. The others include how your site is structured for retrieval-augmented generation, how your content answers questions AI assistants are actually being asked, how much third-party authority you have accumulated on the sources those assistants cite, and how the multi-engine landscape (Google, ChatGPT, Perplexity, Claude) treats your specific sector.

But Cloudflare is one of the two or three levers where a single wrong click has a large downstream effect. That is why the 15 September deadline is worth putting in the diary this week.

If you want a broader view of where your visibility sits across the AI engines that matter, the Optimus AI Visibility audit checks the foundations across the four generative engines UK service SMEs actually need to be found on. We score what is there, flag what is missing, and write the practical fix list. The Cloudflare check is one of the items on that list, and now you have the context to run it yourself.