Build1 distinct publisher3 min readPublished
An embedding scores "the outage is fixed, thanks" much like "outage", because it encodes topic. Twelve extra anchors fixed one developer's test set and halved recall on text it had not seen.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Put the two headline rates into one stream of traffic and they stop meaning what they appear to mean. The 92 and the 76 are per-class rates [6][7], and a badge does not appear inside a class. It appears in a chat where most messages are pleasantries. The source never says how often a message deserves an action, so here is a round assumption of mine: one in twenty. Across 1,000 messages that gives 50 real requests, 46 of them caught, and 950 harmless ones, 228 of which get a badge anyway. Of the 274 badges shown, 228 are wrong, or 83 percent [1]. The author's line about a badge being wrong one time out of four [12] describes the harmless-message rate. What a user experiences is the badge's own hit rate, and in a quiet channel that is the worse number.
That arithmetic also settles whether more anchors were ever going to help. An anchor is one more point in the same space, and the rule is a comparison between two nearest neighbours [4]. "The outage is fixed, thanks" keeps the vocabulary, and therefore the topic, of the tool it matches [8]. Adding "it's fixed" to the anchor list only wins if that phrase pulls on the sentence harder than the outage words do, and the author's own two-minute check says it does not: append "it's fixed, thanks" to positive sentences and the scores barely move [9]. So the remaining lever is the bar, and the bar works inside a band 0.12 wide [2], where "Hello" and "create a ticket" already score almost the same [3]. Twelve anchors and a higher bar looked good on the test set; on unseen messages one real case in two went missing [10][11], roughly six times the miss rate the test set reported [3].
Half the bounding is already in the architecture, and that is the part worth copying. Stage 2 runs only when a user clicks the badge [2], so a false positive costs nothing in tokens and nothing in latency. It is charged entirely against the user's willingness to look at the next badge [12]. That moves the design question from how high specificity can be pushed to what a wrong badge is permitted to do. A passive suggestion survives 24 misfires per 100 harmless messages [7]. Anything that files the ticket itself does not.
Two details make the published numbers optimistic rather than pessimistic. The evaluation ran on 300 generated messages with no users on the system, and the author states plainly that reality will do worse [5], so 76 is a ceiling and 24 is a floor. And the detector splits long messages on punctuation to lift a request out of surrounding politeness, which quietly does nothing when there is no punctuation, as in voice dictation [13]. Generated test text punctuates. Dictated text does not, so the blind spot cannot show up in this score at all.
The deliverable here is not a tuned threshold but a written limit: this stage does not read negation, and nothing downstream may behave as though it does [14].
Ranked by verification strength, evidence, and original report placement.
The worst failure mode is negation: 'There is an outage' asks for an action and 'The outage is fixed, thanks' does not, but they are almost identical to the detector because an embedding summarises a sentence by its topic and both share the same topic.
The author argues a badge that is wrong one time out of four stops being believed: the recall number shines in the demo, the specificity number is paid in production.
The author's conclusion is that the negation failure is a flaw of nature rather than of tuning, and that the right answer is to accept it rather than fix it.
A developer built a small detector that flags chat messages asking for an action, using embeddings; when a message is judged actionable, a badge appears offering to create a ticket or send a document.
The system has two stages: stage 1 is a small model, free to run, that sorts messages; stage 2 is the LLM, called only when the user clicks the badge, because calling an LLM on every message costs money and response time.
A fixed threshold of 0.85 cosine similarity did not work: the model returns scores between 0.80 and 0.92 for everything, and 'Hello' and 'create a ticket' get almost the same score. Raising the bar loses real cases before it loses noise.
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One practitioner's self-reported numbers on synthetic data
The mechanism claims are internally coherent and unusually well instrumented for a blog post: a stated decision rule, a named model, before/after holdout figures, and a reproducible two-minute check anyone can run. But everything rests on one self-published source, a 300-message generated evaluation, no confidence intervals, no independent replication, and no test of other embedding families despite the generalized title claim.
Pre-adoption personal project
Adoption evidence points almost entirely the other way: the author states the project has no users, so a shipped configuration exists but nothing in the sources shows real traffic, downstream users, or third-party reuse of the pattern.
Candid numbers, over-general headline
Slightly overstated on balance. The body is deliberately self-deprecating - it publishes the specificity that hurts, the overfitting result, and the warning that reality will be worse - which pulls toward alignment or understatement. The overstatement is scope: 'embeddings cannot say no' and 'a flaw of nature, not of tuning' are asserted from one small multilingual model on 300 generated sentences, with no test of larger or instruction-tuned embedders and no comparison to classifiers that do model negation.
Self-published developer writing, low commercial pressure
The author is writing about his own unlaunched side project on a developer publishing platform and links to his own earlier 'put the LLM last' argument, so there is reputational and audience-building incentive plus mild self-consistency pressure. No vendor sponsorship, funding, pricing, or commercial relationship is disclosed in the sources, and the post argues against its own initial approach, which is not what a promotional incentive produces.
Moderate on the mechanism, weak on the numbers
Confidence is moderate because the qualitative mechanism (embeddings encode topic, so negation barely moves cosine scores) is well established by the reproducible check and matches the reported behaviour, while every quantitative figure is single-source, synthetic, unreplicated, and internally inconsistent across evaluation sets.
build
SSE in Go breaks twice before your handler runs: an illegal header, then a 30-second timeout1 distinct publisher
build
A cache hit is a quota refund: semantic caching with trigrams and no vector database1 distinct publisher
build
Your LLM cache probably never fires, because it hashes spelling instead of meaning1 distinct publisher
build
Three manual interventions in a month, and every guard was working as designed1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 25, 2026