Cloudflare's new AI training opt-out is backed by network-level enforcement Apple, Google and Microsoft have agreed to honor

Turn the setting on and Cloudflare publishes a `Disallow:` directive in your robots.txt, which is where the name comes from ^[8]. Cloudflare's post also argues that a robots.txt directive alone cannot solve the problem: anyone can publish one, but it cannot identify who is crawling, determine why they are crawling, or stop a crawler that ignores it ^[9]. The rest of the design is network-side. Publish the preference, identify the crawler, classify the purpose, block the ones that ignore it, then report what each operator actually does on Radar ^[10].

This setting exists to avoid blocking ^[12], and Cloudflare's stated reason is that blocking removes a crawler without changing how crawlers behave ^[11]. So for the three named operators, compliance rests on a commitment plus an audit trail: the Accountable designation requires URL-level visibility into which pages were made available for training, metrics showing how content appeared in search, an operator-level opt-out for AI summaries, and assurance that opting out of training will not affect traditional search results ^[13]. Cloudflare says it has been talking to operators directly since July, and that almost all of them agreed site owners should have control and transparency into how their content is used ^[18]. An operator's agreement cannot be verified from outside, but the per-URL training reports can, and I would want to see one before loosening a config that currently blocks.

The demand side offers one comparison: fewer than 1% of Cloudflare sites block search bots, and 17% enable some mechanism to block training ^[4][5]. Training refusers therefore outnumber search refusers by more than seventeen to one ^[6]. Both figures are shares of Cloudflare's own sites, counted as sites. Cloudflare did not publish a traffic-weighted version. For the 17% to say something about the advertising- and subscription-funded web the post points to ^[19], it would have to hold when the count is weighted by traffic or by pages of content. The other 83% of sites have no training block at all ^[7].

Summaries are the harder half. According to Cloudflare, a site-wide yes or no is too blunt, because how much of your content appears in a summary matters as much as whether it appears at all ^[16]. The goal it states for early next year is a single control on Cloudflare for how much of a site's content is included, set once instead of with each operator separately ^[17].