Product1 publisher3 min readPublished
A spoofed AI crawler walked past more than seven in ten sites DataDome tested
DataDome says malicious automated traffic grew 124% in the year to June. Its June test of 21,491 popular sites found that claiming a trusted crawler's name is usually enough to get through the door.
The Product Desk · Product desk
What happened
- Scraping accounted for 70.9% of bad bot traffic and rose 185.2% on the year, making it by far the largest category in the company's mix.
- A June test sent 10 bot types at 21,491 sites across 15 industries from residential addresses in the U.S., Canada and France, and 65.3% of those sites stopped none of them.
- Bots spoofing the identity of trusted AI agents got past more than seven sites in ten, which DataDome attributes to sites trusting the crawler name in the request.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- constraint A publisher that admits crawlers by the name they declare cannot enforce the access terms it thinks it set, because that name is available to anyone willing to type it.
- decision Whoever owns the login page has to decide now which automated sessions are welcome, before the monthly volume makes a blanket block the only option anyone has time to ship.
- exposure Account-abuse controls tuned to stop credential testing will start refusing assistants acting for real customers, and the customer experiences that as a broken login.
- contradiction The report comes from a bot protection vendor counting traffic on its own customers' sites, and the model-training share sits outside what it counts, so scraping dominating the bad-bot mix is the part it supports.
A site that decides which crawlers to admit by reading the name in the request has a control anyone can use. DataDome says identity-based trust is a widespread weakness, because a request claiming to be GPTBot, ClaudeBot or another trusted crawler is often let through on the name alone [9]. Teams call that a crawler policy, and the check that runs at the edge is a comparison against a string they do not control.
The same test also sent a disguised bot that forges the network fingerprint of a real browser. Only 5.5% of sites caught it [8].
Bad bot traffic grew 124% in the twelve months to June 2026, which DataDome puts at more than nine times human traffic growth [2]. Divide 124 by nine and the human baseline grew by less than 14% over the year [1]. Requests that identified themselves as AI agents or crawlers grew 82.3% in the same window, to 52.7 billion across DataDome's customer base [11].
Login pages are where a product owner should look. AI bots sent about 313 million requests to login screens in the first half of 2026, taking 51.7% of all their traffic to login, form, cart, payment and account-creation pages [3]. Monthly volume at login pages went from 11.9 million requests in January to 99.7 million in June [15], roughly 8.4 times in five months [2].
On the training question, DataDome ties some of the growth in scraping to third-party data resellers and agent builders harvesting the web at scale for model training, and says that activity does not always identify itself as an AI crawler [4]. The report does not break the scraping figure down by purpose. Its trillion-request base is DataDome's own customer sites, and DataDome sells bot protection [1]. So the numbers are consistent with training scrapers being the dominant load on public web properties, and they stop short of showing it.
Jerome Segura, vice president of threat research at DataDome, said automated traffic is "growing fast" and is moving into the login, account and transaction flows at the center of the customer journey [16]. Identifying automation is no longer the hard part, he said; the harder question is whether a given session is beneficial or harmful, and blocking everything would cut off the beneficial half [18]. Most defenses in use, he said, are "still largely built around binary choices" [17].
The self-declared name is on neither axis of the grid I would draw. One axis: has this session been verified by something other than what it calls itself. The other: is it touching general content or a login, cart or payment endpoint. Unverified traffic on general content gets a rate limit and a log line. Unverified traffic at a login page gets refused, since name checks did not stop a forged browser fingerprint. Verified traffic on general content gets served against a per-crawler budget. Verified traffic at a payment endpoint is a product decision about whether an assistant may act for a customer, and it is still a small slice: 97.9% of the 29.02 billion AI bot requests in the first half went to homepages and other general content [13]. Verification depends on a short list too. Meta-affiliated bots generated 46.3% of identified AI requests and OpenAI-affiliated bots 34.6% [12], which is 80.9% between them [7].
Most sites are nowhere near that grid. Telecommunications was the weakest industry in the June test, with 82.9% of sites unprotected [10].
What to watch
- Whether DataDome holds the bot mix steady in next year's test, since adding harder types moves the protection numbers on its own.
- Whether Meta- and OpenAI-affiliated crawlers start signing their requests in a way a site can verify without trusting the name.
- Whether credential stuffing, flat this year, repeats the April single-day highs DataDome recorded.