build1 publisher
Thinkingbox grades 507 agent workflows on the backend state they leave behind
Every task in the benchmark is checked against its required terminal state, so any tool order can pass and any extra side effect fails. Claude Opus 5, the strongest model tested, cleared 66.50% at pass@1.
Publishers:arxiv.org
Reality
- Evidence60
- Adoption18
- Hype gap+12
- Incentives35
- Confidence52