Pick an LLM gateway on its cost records, not its model list

The load-bearing part of that argument is the record layout, not the eval harness. The dev.to piece lists nine fields to capture per classification: tenant ID, evaluation-set version, model identifier, input and output token counts, latency, retry count, final status, and the reason for any fallback ^[6]. Seven of those nine the calling application can write by itself ^[1]. The two it cannot are the token counts, which the author hedges as "when available" ^[6].

That hedge is the procurement question in one word. A gateway that returns usage per request lets you keep a tenant ledger. One that returns it per key, or only inside a dashboard, leaves you estimating a tenant's spend from document length and then defending the estimate to an account team that has already noticed the bill ^[7]. The author's stated failure mode is a monthly aggregate concealing either one tenant with unusually long documents or a retry storm ^[7]. Both are invisible in a total and obvious in a ledger.

The same asymmetry runs through the interface he keeps between adapter and application. ClassificationResult carries seven fields, and only two of them, score and rationale, are the classification ^[10]. The remaining five are there so someone can reconstruct which tenant, which model, how many tokens, and why the call fell back ^[2]. He is explicit that the model call is meant to stay replaceable while the audit record does not ^[12].

Cost arrives late in the routing policy, and that ordering is the actual advice. Gate one removes models that cannot satisfy the output shape or the data-handling requirement. Gate two removes anything that failed the hold-out evaluation. Only the survivors get sorted on bounded cost and latency, and the fallback has to have cleared the same gates ^[13]. In his sample code the eligibility test is JSON support, quality at or above 0.94, and p95 latency at or below 2500 ms, with cheapest-of-the-eligible as the final line and a raised error when nothing qualifies ^[14]. Price is the fourth question ^[3].

The validation point is narrower than it looks. A structured response that parses can still carry a score outside the rubric range or a rationale that discloses personal data nobody asked about, which is why the parser sits between the provider and the hiring workflow rather than after it ^{[16] [15]}. Normalising a request does not make a vague rubric precise ^[9]. His own guard is blunt: reject any score that is not an integer from 0 through 100, reject a blank rationale ^[11].

One caveat on weight. This is a single practitioner's design argument, referencing OpenAI's function calling guide for the general validate-the-arguments pattern ^[8]. It contains no measured comparison of named gateways on contract stability, so the ordering it proposes is a hypothesis you would have to test on your own labeled queue ^[4].