build1 distinct publisher
A refactoring benchmark stops the best agent at 41.2%, and the tests are the story
SWE-Bench ProMax puts frontier coding agents on 170 curated refactoring commits. The number that should move procurement is a different one: nearly 60% of unsolved SWE-bench Verified tasks have flawed tests.
Publishers:thenewstack.io
Reality
- Evidence38
- Adoption14
- Hype gap+12
- Incentives62
- Confidence42