Featherless open-sources Simple Jev to make open models pick labels without writing text

"Simple Jev never 'writes' an answer: we stop the model right where it would choose, read its scores for each allowed option, and output the probabilities," Cheah said ^[5]. His longer name for the design is a "shared-prefix, two-stage, prefill-only, logit-based classifier" ^[6]. Taken with the first quote, prefill-only means the model reads the prompt and the library stops at the point where the first output token would be chosen, so no decoding loop runs ^[5]. Featherless does not bill output tokens ^[20]. Under this design there are none, only a probability for each option the caller listed ^[5].

The design is sound, and Cheah does not claim it is new. He said "classifiers aren't new; it's what universities taught for AI before ChatGPT. What's new (and credit to Jev here) is a well-designed zero-shot API for it" ^[16]. The strong part is the constraint. The library scores only the labels the caller supplies, so the answer is always one of them ^[5]. There is no free text to parse and no malformed reply to retry. Cheah also said "Any graduating AI/ML PhD could reimplement Simple Jev from one sentence" ^[7]. It is an unusual line for a launch, and he followed it with a forecast of "hundreds more open source Jev clones" ^[8].

Cheah put a typical decision at 500 to 1,200 tokens and said that at the $0.03 floor it comes to about $15 to $35 per million decisions ^[13]. "It's almost too cheap to meter in a literal sense," he said ^[14]. The floor is a beta price that may rise ^[12]. Featherless's own listings put its Qwen-based classifiers at $0.28 and $0.30 per million input tokens ^[21]. At those rates, Cheah's token range costs $140 to $360 per million decisions ^[1].

The article states that skipping conversational text cuts latency and compute ^[17]. It does not include latency figures, accuracy against a frontier model on the same inputs, or a count of how many production calls are classifications at all. Cheah's per-decision figure carries over to another team only if its inputs stay near his token range, the floor-priced model is accurate on its labels, and the beta price holds ^[12][13].

Cheah concedes that large closed models can do this work but calls them "the wrong tool for the job," because they get there by generating text through "enormous multi-trillion-parameter LLMs" ^[19]. "Sure, the pizza tank gets there eventually, but it's slow, it's expensive, and it's the wrong vehicle," he said ^[15].

His example workload is classifying a support ticket ^[18]. For a ticket router with a stable label set, I think the design is right. The per-label probability also gives a cut-off for sending low-confidence tickets to a larger model ^[5].