Latency spread narrowed to 38ms from 2353ms when typed questions replaced generated prose

Two typed questions in one request do not chain. TypeSafe AI's guidance on composing questions says questions in a single request run in parallel and cannot see one another's answers, and it advises keeping policy in code and raw judgments reusable ^[17]. The team sent two: which category the account belongs to, and what the operator should do about it ^[14]. The action question had no verdict to read. It guessed from the raw state every time, and the guess leaned hard toward the severe option ^[18].

The team writes that an incoherent answer was useful precisely because nobody can talk themselves into it ^[23]. Their fix is the one already in the vendor's guidance: derive the action in code. `deriveAction` takes the verdict and the confidence, returns `suspend` for spam or phishing at 80 or above and `hold_sends` below it, and `watch` for suspicious or unclear ^[20]. The text model they already run never produced that contradiction, because it writes the verdict and the action in one pass, so the action comes after the verdict. The post says that removes this particular failure without guaranteeing consistency ^[19].

Almost none of the 7x saving is token price ^[5]. Per input token the two models are 1.31x apart ^[6]. Output is free on the new one, and output is 83 percent of what they pay now ^[7]. Divide the remaining 17 percent by 1.31 and the new bill lands at about 13 percent of the old, a 7.7x reduction, close to the 7x they report ^[3].

Pressing the button gathers volume counters, the domains the account mails, the link hostnames in its messages and a dozen recent subject lines, then awaits the model inside the request handler ^[11][12]. Message bodies never reach the model ^[11]. At the median the new model was 12x faster, and closer to 3x once the team discounted the paragraphs their old model writes that nobody reads ^[3][4], so roughly a factor of four of the median gap was output length ^[2]. The spread they measured was 38ms against 2353ms ^[8], about 62 times narrower ^[5].

Their baseline was a mid-size open model on a serverless inference endpoint, and Jev is not a frontier model either ^[13]. The vendor's published figure was 193.6x; a 12x median is about 16 times smaller ^[1][3][1]. The post lists three requirements for getting a number of your own: your own workload; somewhere to run the test near the thing being tested, not a laptop; and a way to tell whether an answer was right that is not another model's opinion ^[21]. Typed output also removed a defect the prose model produced at three replies in fifty, six percent, on a route where a parse failure renders an error page ^[9][4]. Before any of this, the team produced two confident wrong numbers: the parallel-questions error, and scoring a feature against accounts it had never seen ^[10].