Reversing the urgency options moved "not urgent" from zero picks to 250

The lowest urgency level was the gold label on 77 of the 300 Japanese emails, about a quarter of the set, and the checkpoint never predicted it once ^{[10] [2]}. Three explanations fit that: a degenerate ordinal head, a model reading the slot instead of the content, or a bad Japanese phrase for that level ^[11]. Telling them apart took one more pass over the same 300 emails under five option schemas, including a reworded low/medium/high set and a four-level variant ^[12]. On the reversed pair, "not urgent" went from 0 picks to 250 with the emails and the wording unchanged ^[13].

Two controls close the replies a busy maintainer reaches for first. The harness was rebuilt from the two lines in the README, `laya.load()` and `agent.predict()`, with no wrapper in between; against the saved probabilities the maximum difference was 2.2e-16 on raw values ^[14]. The English control reused the same label weights, the same generator and the same rejection rules across 290 emails and five conditions, in a language the checkpoint was trained on ^[15]. That is 1,450 inferences, and the first-listed option never won one of them ^[3].

For any of that to say something about your workload, the generator has to be beside the point. These emails are synthetic: labels fixed first, a local LLM told to write text matching them, and any email containing the label words discarded and regenerated ^[5]. Both language runs used laya-multilingual, and the report does not include results for the English-only laya build ^{[4] [15]}. The damage is also confined by question type. The 4-way department question scored 0.747 against a 0.380 majority baseline in the same run, while the cancellation question came in at 0.543 against a 0.703 baseline with an AUROC of 0.523 ^{[7] [9]}.

From issue #131 at 05:56 UTC on Sep 22 to the merge of PR #259 at 17:48 the next day is 35 hours and 52 minutes ^{[16] [21] [1]}. Not all of that was the reporter's work: a third party, AlKor13, traced the behaviour to the weights at 07:15, and the labelled A/B test of that hypothesis followed at 10:56 ^{[17] [18]}. v0.3.7 shipped with the limitation in the README 10 hours and 45 minutes after the maintainer's decision ^{[19] [20] [5]}. The retrain has not happened ^[24]. The author wrote that what he contributed was "a measurement nobody could wave away, and a tool that says whether the fix worked" ^[27].

One comparison stayed unrun. laya benchmarks itself against TypeSafe's Jev, a hosted API whose customer agreement prohibits using the service or its outputs to develop similar products, and the author is building sokudan, a Japanese model in the same space ^{[26] [25]}. The developer with the strongest reason to check that comparison is the one the licence excludes.