Build1 distinct publisher3 min readPublished
A CDN rule aimed at AI crawlers matched the Vendor/Language User-Agent shape that the official OpenAI and Anthropic SDKs send by default. The harness meant to catch it sent an allowlisted string of its own and reported green.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
A User-Agent shape rule has one honest signal behind it. `Vendor/Language version` means a programmatic client, and the official SDKs are programmatic clients, because that pattern is what they send by default [4]. The rule did what it was written to do. What made it hard to see is where it sits: the 403 is returned at the edge, before the request reaches the application at all [7], and browser traffic never carried the matching shape, so the audience most likely to complain in words was never blocked [6].
The suite itself is decent engineering and worth saying so. Rather than installing nine GUI clients and clicking through them, the team reproduces the characteristic HTTP request each client sends, fires it at the real user-facing endpoint, and checks the response field by field [10]. Its assertions covered streaming frames, tool-call assembly, `response_format`, and the Anthropic event sequence [11]. Each of those answers whether the server understands the request rather than whether the request is admitted [11]. The author's own diagnosis is the useful one: the test client's identity happened to be the safe one, so the suite kept passing for a reason that had nothing to do with test coverage [13]. The safest identity in the stack turned out to be curl's default, which nobody picked on purpose [20].
The check that finds it is two requests with the same body, path and key, differing only in the User-Agent; if the statuses disagree, something in front of the application is filtering on UA [14]. Count the matrix as described and it is small: one curl baseline against the API and one against a static asset, plus the three SDK defaults and the five crawler strings, is ten requests [22]. The crawler half runs against a static asset rather than the API [16], which is the right call, because the thing being tested is reachability of the file, not the endpoint.
Two caveats on transferring the numbers. The author offers their own `/llms.txt` answering 200 to all five crawler UAs as of the retest date, 2026-08-31, as a live target for validating a harness [17][18]. That is a statement about one edge configuration on one day, and it proves exactly one thing about your setup: that your harness sends the header you think it sends before you point it at your own host. Second, keep the real-SDK confirmation step, because a hand-typed `OpenAI/Python 1.68.2` is a guess pinned to a version, and the installed client moves without asking you [15][5]. Match the block on its signature too: status 403 plus a `server: cloudflare` header [19].
Duration is the one number missing from the post. It says the rule ran that way for a while, with the suite green throughout [21][2], which is the sort of gap that only exists because nothing downstream of the edge had anything to record.
Ranked by verification strength, evidence, and original report placement.
A bot-blocking rule written at the CDN edge matched the official OpenAI and Anthropic SDKs by the shape of their User-Agent string and returned 403 to all of them.
The team's compatibility test suite reported green the entire time the rule was in place.
The suite was green because the probe sent its own User-Agent, and that User-Agent was on the allowlist.
The rule was aimed at AI crawlers and matched on a Vendor/Language version User-Agent pattern, which is what the official SDKs send by default.
Default SDK User-Agents given as examples: OpenAI/Python 1.68.2, OpenAI/JS 4.104.0, Anthropic/JS 0.120.0.
Humans were unaffected: someone clicking a link from an external site got a normal page, because browser traffic never matched the pattern.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 31, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
product
Four leaderboards, four denominators: what you buy when you standardize on a coding agent1 distinct publisher
product
A 2x LLM bill is not a bug report: token spend is an observability problem1 distinct publisher
build
Every one of thirteen named 2025-26 incidents ran on a credential that still worked1 distinct publisher
build
Your Multi-Key Failover Is The Most Expensive Line On Your Coding Agent Bill1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
First-hand, single shop, partly re-runnable
The mechanism is nailed down about as well as one team can nail it: a 403 from the Anthropic SDK against a 200 from curl with only the User-Agent differing is a clean single-variable isolation, and the prescribed check is published as runnable commands. What holds the score down is that the same people who wrote the rule are the only witnesses to every number, and the one figure that would size the harm — how long the block was live — is left as 'a while'.
One gateway, its own scorecard
The only deployment in evidence is the author's: a Cloudflare rule changed, a nine-row client matrix re-run, a handful of crawler User-Agents re-probed against llms.txt. That is real, dated activity rather than a proposal, but it is one host measuring itself, and the recommended two-request check has no reported uptake anywhere else.
Sold slightly below what it shows
A post that volunteers five straight vision failures and a 503-instead-of-4xx error-shape bug is not inflating itself, and the headline finding is smaller and more useful than a general essay on testing would be. The one place the reach exceeds the reporting is the aside that this blind spot sits in most home-grown test tooling — that rests on a sample of one, as does the confident story about developers blaming their own API key and closing the tab.
Self-published, self-implicating, mild plug
The author is both narrator and defendant, which usually argues for discounting — except the story is 'our edge config silently rejected the SDKs our product exists to serve', and it is followed by a vision column the author refuses to claim. The commercial edge is soft: readers are pointed at the team's own /llms.txt as a test target, which doubles as traffic and as a standing invitation to catch them regressing.
Firm on the mechanism, thin on the scale
We would bet on the causal chain — an allowlisted probe UA cannot detect a rule keyed on UA, and the curl-versus-SDK comparison closes it. We would not bet on anything about magnitude: one publisher, one self-measuring host, one retest date, and no way from here to tell whether this was a two-hour misfire or a two-month leak.