Pick the Routing Pattern That Can Defend a Tenant's Invoice

A dev.to post on a marketplace SaaS chatbot that reviews code changes puts the architecture decision in an unfashionable place: choose the runtime pattern that records tenant, route, model, token usage and fallback reason in one usage event, because a single credential and automatic fallback are useful but neither matters if a team cannot explain each tenant's bill ^[1]. That moves the build question from provider feature lists to the accounting boundary, which the same author frames as who must own normalization, failover policy, and evidence when a customer questions a charge ^[3].

Treat "one key" as a deployment property rather than a selection criterion: it cuts secret distribution across services, but it does not prove that fallback is safe or that per-tenant charges are reconstructable ^[4].

The three candidate patterns sort by who holds the burden. A direct integration preserves maximum control, a self-hosted gateway centralizes routing while keeping operations in-house, and a managed broker transfers more of that operational load ^[2]. Direct clients fit when the review workflow depends on provider-specific capabilities or only one provider is likely to stay active, at the cost of adapter work that drifts independently across auth, timeouts, usage extraction, error classes and schema validation ^[9]. A gateway fits when several teams need the same policy and shared services already exist; LiteLLM is cited as one open-source example documenting a proxy-oriented approach, and someone still owns its availability, configuration review and telemetry pipeline ^[10]. A managed broker fits when minimizing control-plane operations outranks owning routing detail, which makes the contract's usage fields, model identifiers, export path, regional handling and throttling behavior the thing you actually buy ^[11].

The test the post proposes is the part worth stealing. Run one fixture against all three patterns, carrying tenant ID, repository ID, pull-request ID, a fixed prompt version and a JSON output schema ^[5]. Force a normal response, a 429 rate limit, and a timeout at the application deadline; the run passes only if the final finding stays tied to the original tenant and every attempt is visible, with no merging of attempts into one opaque total ^[6]. As the author puts it, a fallback can produce a good answer while leaving finance with a bad ledger ^[7].

Per attempt, that means tenantId, requestId, attempt, provider, model, input and output token counts, timestamps, outcome, fallback reason, plus prompt and schema versions, which is how you later explain why two similar diffs consumed different resources after a rollout ^[14]. Monetary values get written only after applying the rate card effective at the attempt timestamp; token counts are evidence, a mutable current-price lookup is not ^[15]. Metering rebuilt from application logs after the fact fails, because logs are optimized for diagnosis while a ledger needs stable identities, explicit units and append-only attempts ^[13]. Tenant identity comes from trusted server-side authentication ^[18].

Why the model field is load-bearing: a separate dev.to framework, whose 2026 model names and prices read as projections rather than a live rate card, lists input pricing from $0.14 per million tokens to $10.00 ^[19], a spread of roughly 71x ^[20]. A silent hop to a pricier tier is a billing event, not a retry.

Watch fallback rate by tenant tier over a window rather than alerting on a single fallback, since one success is expected behavior while a sustained shift signals throttling, a policy change or a workload change ^[17]. If you are evaluating a broker, the author's position is that no generic feature checklist settles the trade-off and that a replayable tenant fixture plus an exported usage sample is the better evidence ^[12]. Ask for the export before signing.