project

themsquared/jev-benchmark

Open-source, standard-library Python harness for testing typed-decision models on hand-labelled agent tool-call risk classification cases, shipping its task file and raw per-call results alongside the code.

Known aliases

jev-benchmark

Relationships

No evidence-backed relationships are recorded.

Current clusters

build1 publisher

A 60-case benchmark puts Jev's usable confidence threshold at exactly 1.000

TypeSafe AI launched Jev with 193.6x and 444.6x multipliers and nothing to re-run. An independent harness measured something else, whether the confidence score is calibrated well enough to route escalations on.

Publishers:dev.to

Reality

Evidence66
Adoption14
Hype gap+38
Incentives48
Confidence58