Supersonic Labs trained Julia 1 for $104 on cloud GPUs, then benchmarked inference on a laptop CPU

On each call, the caller sends a context, a question and its own candidate labels. It gets back a score for each label, in the order it sent them ^[3]. The integration code is an argmax plus an index into a list the caller already holds. A generative model doing the same job streams text such as "The correct category is: billing", and the caller then has to parse the label back out ^[19]. Julia 1 never generates text, so it cannot return a malformed answer. It can still return the wrong label ^[5].

The same interface covers three question types: pick one option from a closed list, place the input on an ordered scale, or give the probability that a statement is true ^[4].

The $104.08 is the lab's cloud GPU bill for training and experiments ^[2]. An adopter does not pay it. The weights are on Hugging Face under Apache 2.0 ^[1], so the running cost is CPU time. The bill is small because of a design choice I think is correct. According to the dev.to write-up, the team lacked the data, infrastructure, time and money to train a multilingual model from scratch. Instead it took mmBERT-small, a multilingual encoder from Johns Hopkins CLSP, kept its tokenizer, and trained an added decision head to score supplied options ^[15].

Quality held on CPU. The GPU evaluation scored 73.15% on 2,000 typed decisions ^[6]. A September 25 re-run on a plain CPU scored 72.55% ^[10], 0.6 points lower ^[1]. An Apple M4 took a median 33.15 ms per decision one call at a time, about 30 a second ^[3]. Batched 16 at a time, it reached about 51 a second ^[11]. On an Intel Core i5-1235U the median was about 295 ms ^[11], roughly 3.4 sequential decisions a second ^[7]. The Samsung SM-X510 tablet ran at 203 ms through ONNX Runtime, with the 550 MB weights memory-mapped and peak process memory at 393 MB ^[13]. RuntimeWire reports five decisions a second on that tablet, running on CPU after the accelerator path fell back ^[14]. That is consistent with 203 ms per call ^[2].

The lab documented the failure itself. Each native call accepts 2 to 20 options ^[3]. Banking77 has 72 labels, so Julia 1 sends them through a Router that narrows candidates in groups before the final pick. That narrowing can discard the correct answer ^[9]. If an early round drops the right intent, the final pick has no correct option left to choose. Julia 1 got 64 of 100 against an 87% reference ^[8], 23 points behind ^[4].

The wins deserve the same scrutiny as the loss. On a 100-example pilot each example is worth one point, so the 38-point emotion gap (86 of 100 against 48%) rests on 100 cases ^[7][5]. The typed-decision margin over Jev is 0.45 points, or 9 of 2,000 cases ^[6]. Jev is another small decision model in the same category ^[6]. The published evaluation does not include a head-to-head against a generative LLM on the same labels. RuntimeWire describes the pilots as company-reported results, not evidence of performance across production workloads ^[18]. The lab's page lists the limits: no external knowledge, no multi-step reasoning, and 100-example pilots that are signals, not guarantees ^[16].

These numbers transfer only under three conditions. A team's labels have to fit in 20 options or survive the router. The labels have to be distinct from one another. And 200 to 300 ms per decision on low-end hardware has to fit the latency budget ^[11][13]. The lab's own example, routing a ticket to billing, shipping or access ^[4], meets the first two. For label sets like that, I think a trial against an existing LLM classifier makes sense. According to the dev.to write-up, Supersonic's planned hosted API costs $0.025 per million input tokens with no output charge, against $0.042 for Jev ^[17].