benchmark

Code Review Bench

Open benchmark that scores AI code review bots on real GitHub pull requests, counting a suggestion as correct only when a developer acted on it, and reporting precision, recall and F1 per tool. Its code is published under the MIT licence.

Known aliases

code-review-benchmark

Relationships

No evidence-backed relationships are recorded.

Current clusters

build1 publisher

Martian scores code review bots on whether a developer actually changed the code

Code Review Bench reconstructs the timeline of 16,017 open source pull requests and publishes precision and recall beside every F1. The top five tools sit within five points of each other, on samples of very different size.

Publishers:dev.to

Reality

Evidence66
Adoption45
Hype gap+12
Incentives55
Confidence55