Loading today’s stories
Build1 distinct publisher3 min readUpdated
A FastAPI writeup on isolating Anthropic keys per tenant is really a bug report about ambient request context, and about what breaks the moment a background task reads it.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
A post on dev.to describes CitizenApp arriving at 15 tenants with one global Claude API key, and the author's conclusion that a single customer's agentic loop could burn the shared quota and throttle everyone else [1][2]. The interesting part is not the quota arithmetic but where the author says the previous design actually failed: background tasks that queried Claude after the request had moved on [7].
The earlier arrangement will look familiar. A `get_current_tenant()` middleware set `request.state.tenant_id`, and handlers then fetched the API key themselves and passed it down the call chain [5][6]. According to the author, once background tasks started calling Claude, context vars leaked, rate limits went unenforced, and working out which tenant a given call belonged to took hours [7]. That is the predictable end state of ambient context: middleware runs once per request, so the tenant's key has to be parked somewhere a later reader can find it, and shared rate-limit buckets get touched on the hope that concurrent requests do not collide [8][9].
`Depends` changes the shape of the problem rather than the amount of code. Dependencies resolve per request, compose into other dependencies, and can be cached per dependency with `use_cache=True` [10]. The practical difference is that a handler receives a client object it can hand to a background task explicitly, instead of a task reaching back for request state that may no longer be the right one [4].
The implementation is small. Tenant rows carry the Anthropic key, which the author notes should be encrypted in production, a `max_requests_per_minute` defaulting to 60, and a `preferred_model` defaulting to claude-3-5-sonnet-20241022 [11]. `get_tenant_id` decodes a bearer JWT and raises 401 on a missing or invalid token [12]; `get_tenant` loads the row once per request and raises 404 [13]; `get_claude_client` and `get_rate_limit_bucket` are kept as separate providers so either can be injected without the other [14]. Note that this puts a database lookup in front of every model call, and the per-dependency cache only deduplicates within a single request [13][10].
Two things in the listing deserve flagging. The post opens with "no globals", but the clients and buckets live in module-level dicts keyed by tenant id [3][15][3]. And the buckets are in-memory sliding windows, which the source itself says should be Redis for distributed deployments [16]: run four workers and the enforced ceiling per tenant becomes 4 x 60 = 240 requests a minute, not 60 [1]. The client cache has a subtler edge. `get_claude_client` only constructs a client when the tenant id is absent from the dict, so a key rotated in the database is not observed by a warm process until that process restarts [17][2].
What to watch: whether the bucket store moves to Redis before the worker count changes, because the limit multiplies silently [1]; whether the per-tenant client dict acquires an invalidation path for key rotation and tenant offboarding [2]; and whether background tasks are actually handed the injected client as an argument, since that is the exact case that broke the middleware version [7].
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
The stated fix is to use FastAPI's dependency injection system to make tenant-specific Claude clients and rate-limit buckets first-class citizens, with "no globals, no thread locks" and no detective work about who is using the API key.
The post states that middleware runs once per request, so the options are to parse the tenant ID from the request, look up the key, and store it somewhere accessible such as request state, context vars, or thread-local storage.
The other middleware option described is to hope that concurrent requests do not collide when accessing shared rate-limit buckets.
The post states that FastAPI dependencies are resolved per request, or per dependency cache if use_cache=True is set, and that they compose naturally, so a handler declares what it needs rather than knowing how it is built.
The Tenant SQLAlchemy model has columns for name, anthropic_api_key (with a code comment saying it should be encrypted in production), max_requests_per_minute defaulting to 60, and preferred_model defaulting to claude-3-5-sonnet-20241022.
The get_tenant_id dependency takes bearer credentials via HTTPBearer, decodes a JWT with HS256, and raises HTTP 401 if the tenant_id claim is missing or the token is invalid.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Code artifact is inspectable; operational history is not
The implementation claims are strong evidence of the artifact itself: models.py, dependencies.py and routes.py are reproduced, so the Tenant schema, the JWT/HTTPBearer chain, the module-level caches and the bucket algorithm can all be read and reasoned about directly, which is also what makes the two derived defects verifiable. Everything outside the code is a single unverifiable first-person account: no traces of the leaking context vars, no before/after latency or throttling data, no test or load-test output, and the final listing is truncated mid-handler so the background-task case the dek turns on is never actually shown.
One self-reported small deployment
The only adoption signal is the author's own disclosure of a 15-tenant product calling the Anthropic API, and even there the described dependency-injection design is presented as a fix being adopted rather than as a pattern with measured production history. No other teams, repositories, downloads, or third-party deployments appear in the material, so this scores as a single unverified datapoint rather than absent evidence.
Framing overshoots the shipped implementation
The prescription is sold as eliminating globals, thread locks and unenforced tenant limits, but the accompanying code keeps two process-level global dicts and an in-memory bucket that enforces the configured ceiling once per worker, so a multi-worker deployment permits several times the intended per-tenant rate. The cached-client path also silently retains a rotated key. The gap is moderate rather than severe because the core dependency-injection mechanics are real, correctly described, and the post does flag Redis and key encryption as production work.
Mild self-promotional, self-published
This is a self-published developer-platform post whose framing centres the author's own product, CitizenApp, and the author's preferred pattern, with no editorial review layer and no disclosed sponsorship, vendor relationship, or commercial ask visible in the material. The incentive to present the refactor as a clean success therefore exists and plausibly explains the unqualified "no globals" framing, but nothing in the sources indicates paid placement or a vendor-aligned agenda.
Confident on the code, thin on everything else
Confidence is high for claims about the artifact because the source ships its own code, and the two derived defects follow deductively from that code. It is low for the operational narrative, the scale claim, and any generalisation about how well the pattern works in production, because the cluster contains exactly one self-reported publisher, no corroboration, and a truncated body that omits the background-task path the story is framed around.
build
A RAG stack lived seven hours before a hosted embedding endpoint returned 4041 distinct publisher
build
Force the tool call, then hand Lightsail a long-lived key1 distinct publisher
build
Your 90% Cache Hit Ratio Is a Lagging Indicator. Alert on Cold Misses Per Key1 distinct publisher
build
Three services you can delete: queue, cache and search in one Postgres1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 14, 2026