Calibrate once, route forever: the cascade arithmetic says the cheap path carries under half

Whatever the proxy handles, the oracle handles the rest, and on the court-opinions example that rest was 59.4% of records ^[1]. Across the three worked examples in the post the proxy's share averages 47.5% ^[2]: 45% on a toy binary task ^[7], 57% on open-ended extraction ^[8]. The expensive model stays in the loop. What changes is the fraction of traffic that reaches it.

Calibration is paid for out of that fraction. Five hundred oracle labels ^[3] are 10% of a 5,000-record job, which puts oracle-priced calls at 69.4% of volume and caps the saving at 30.6% even if the proxy were free ^[5]. Run the same task over 50,000 records and the sample is 1% of volume, with a ceiling of 39.6% ^[4]. The technique is a volume play, and below a few thousand records the sample is the bill.

Then there is the overshoot. The court run was configured for a 0.9 target and returned 0.976 ^[6], which is 7.6 points of accuracy above what was asked for ^[3]. The accuracy mode is one-sided by design: match the oracle on at least the target share of records, using the proxy as often as possible ^[4]. Accuracy above target is not a bonus, it is threshold conservatism, and it is charged at oracle rates. For binary work the package also offers precision and recall modes that fix the oracle budget instead ^[5], which is the honest way to buy a known amount of escalation.

The most useful line in the post is a failure mode rather than a saving. If the calibration run shows the proxy handling only a small fraction, the post's reading is that the logits are not tracking the oracle, and the fix is a different proxy or a simpler prompt before any tuning ^[13]. That makes 500 oracle labels a cheap test of the premise itself: either the cheap model's token probabilities correlate with the expensive model's answers on your data, or the cascade is not available to you at all.

That correlation sits on a vendor field. The example passes logprobs and top_logprobs to ChatOpenAI and reads the scores back out of response_metadata on gpt-5-nano ^[15], with the confidence built from the probabilities of the response tokens ^[2]. The post's own caveat on the court figures, one run, subject to model versions, API behaviour and dataset changes ^[9], applies to the threshold as much as to the accuracy. The claimed advantage over FrugalGPT and SUPG is statistical: guarantees that hold at any sample size, with adaptive sampling and better estimation ^[12]. A guarantee about a sampling procedure is not a guarantee about a provider changing what the logprobs mean.

The paper's headline figure is a margin over competing methods rather than a discount on an invoice ^[10]. The follow-up's further 48.5% average comes from rewriting prompts into simpler surrogate questions, reading only relevant document chunks, and searching for the cheapest cascade ^[11]. The first two of those would cut a single-model pipeline's costs too.