build1 publisher
Running every AppWorld task five times drops a ReAct agent from 77% to 53%
IBM Research calls the 24-point drop between passing once and passing five times the consistency gap. Independence would have predicted a 27% five-run rate, so the failures are clustering on particular tasks.
Publishers:dev.to
Reality
- Evidence45
- Adoption20
- Hype gap+12
- Incentives45
- Confidence48