Skip to content

Build1 publisher3 min readPublished

A classifier with no text generation pushes the runtime check out of the prompt and into code

Clusterflick sorts 450 to 600 unmatched London cinema listings a day into ten categories. Its maintainer spent an evening rebuilding that call around typed questions and exclusion lists, with the feature-length comparison done in his own code.

The Engineer · Build desk

What happened

  • Clusterflick pulls listings from more than 400 London venues, and the 450 to 600 a day that do not match a film in The Movie DB need sorting into one of ten categories.
  • For the past year a language model has done that sorting, taking title, runtime and description in a prompt and returning JSON with a category, a confidence score and a one-line reason.
  • Jev takes structured state and a set of typed questions and returns a probability distribution over the possible answers, with no text generation anywhere in the loop.
  • The rewritten call asks four yes/no questions alongside the category question, and they return in the same round trip at no extra cost because every question in a request is evaluated in parallel.
  • The reason field goes away, along with the caution the maintainer had gained by making the language model show its working before it answered.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • capability Facts that used to be buried in prose the maintainer never read are now separate answers he can log, threshold against the category, or flag when they disagree.
  • decision Every numeric judgement has to be settled before the call, so a boundary like feature length becomes a code path with a number in it instead of a line of prompt text.
  • constraint Demanding an explanation is no longer an option, so a wrong category arrives without the reason line that used to give the maintainer a check on a bad label.
  • cost The migration cost sits in the taxonomy, not the call site: the ten option descriptions have to be re-expressed as structured objects before the first request is worth sending.

On its way to a label, the old prompt was settling four separate questions: whether there is a feature-length film in the listing, whether there is more than one, whether this is television, and whether a film is actually screened or only discussed [10]. Those four are now asked directly, and they land exactly where Jev's documentation says it is weakest, on counting and runtime arithmetic, where asking each one narrowly is the recommended handling [12].

The reason the category descriptions had to be rewritten is in the docs. Jev "answers the question you wrote, not the one you meant", with scoping words and negations read at face value [13]. Several of the ten categories are defined mostly by what they exclude: a concert film belongs in `movie` and not `music`, and a film with a Q&A afterwards is still a `movie` [14]. A language model usually infers that from a loosely worded description, and a buried "not a comedy film screening" clause is less reliable here than the same exclusion given a field of its own [15]. Each option now carries an explicit `excludes` list, and the documented advice is to reach for it only once two options are similar enough that the model keeps confusing them [16].

Numbers go out of the request entirely. Jev is documented as not being a calculator, and it does better on semantic language than on numeric formats [17]. So instead of handing over `Duration: 73 minutes` and hoping, the maintainer does the comparison in his own code and passes the result as a fact, worded deliberately without naming a category so that it states the relationship [18].

Some of the daily volume is just title drift. The post's example response classifies "Star Wars: Episode VI Return of the Jedi (1983)" as a `movie` with a confidence of 9 [6]. That listing only needed categorising because The Movie DB calls the film "Return of the Jedi" [7].

The write-up documents the redesign and does not include an accuracy or cost comparison against the prompt it would replace [24]. That matters for anyone weighing the same swap, because the incumbent has a year of hardening behind it: at 450 to 600 listings a day, roughly 164,000 to 219,000 decisions [4][23]. Against that, the experiment cost an evening [9].

Some of the residual error will stay put whichever engine wins. The listing that opens the post is the quarterly meeting of the UK Buster Keaton Society at the Cinema Museum in Kennington, running 4pm to 7pm, with members requesting beloved gems to watch together [20]. Reviewing that batch by hand, the maintainer marked one listing as `multiple-movies` and gave up on the next, recording it as having no single right answer [21]. "Even for a human this is a hard problem to solve," he wrote [22].

What to watch

  • An accuracy comparison on the same listings between the Jev configuration and the prompt hardened over a year.
  • Whether the excludes lists stay confined to confusable pairs or spread across all ten category options.
  • Whether the four yes/no answers get used as a gate, so a listing whose answers contradict its category goes to human review.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories