AI-agent requests on Cloudflare's network grew more than 1,700% in a year

Cloudflare's average request rate rose by a factor of about 1.8 from the end of 2024 ^[1]. Daily agent requests rose by a factor of at least 18 over the past year, since an increase of more than 1,700% is 18 times the starting count ^[2]. The two windows differ, and the slower figure covers the longer period. Agent requests are growing roughly ten times as fast as the traffic around them, and the window mismatch makes that gap an understatement ^[3].

For an operator, the shape of that load matters as much as its size. A training crawler collects a site's pages to build a model ^[10]. An agent fetches pages on a person's behalf, often because that person asked a chatbot something ^[15]. It comes back each time someone asks about the same content ^[10]. Cloudflare's contrast is that agent traffic grows with how many questions people ask, not with how much a site publishes ^[10]. I think that puts agent load in the same capacity plan as human load, with its peaks set by demand inside someone else's product.

Cloudflare's own advice has moved with its data. "Our first instinct was to block all automated traffic," the company wrote ^[8]. The same post argues that a blanket refusal is not nuanced enough for the Internet economy now being built ^[7].

Each figure here is an aggregate from Cloudflare's network ^[1][2]. The claim that more than half of Internet traffic was not human this year is Cloudflare generalising from that vantage ^[3]. For it to describe a particular origin, that origin's visitor mix would have to resemble the traffic Cloudflare handles in aggregate. The 40% fall in human traffic is a ceiling. Cloudflare reports it as "as much as" 40%, in less than a year, for the most heavily crawled categories ^[4].

The piece of this I would adopt first is Web Bot Auth. "A bot's name is only worth something if you can trust it," Cloudflare wrote ^[16]. An impersonator can copy a name but not the operator's cryptographic signature on the request ^[12]. Moving bot identity from a claimed label to a verifiable signature is good engineering. The dashboards for reading all of this, AI Crawl Control, Business Insights and BotBase, are Cloudflare products ^[13]. The company counting the bots also sells the counter. The post does not include a per-site distribution of the traffic mix.