Martian scores code review bots on whether a developer actually changed the code

The F1 column is not an independent measurement. Cubic Dev AI's 72.9 precision and 59.8 recall give 2 x 72.9 x 59.8 / (72.9 + 59.8) = 65.7, the F1 printed beside them, and the same harmonic mean reproduces the published F1 on every row where all three numbers appear ^[21][25]. Read the two input columns and you have the row. Read F1 alone and you cannot tell whether a tool got there by commenting rarely and well, or often and roughly.

Everything on the board rests on what "acted on" means. Martian reconstructs each review timeline from bot suggestion to developer response to the code change that landed, and a suggestion counts as a hit only when a developer acted on it ^[5]. A comment that correctly flagged a race condition and got ignored is a false positive under that rule. Precision here is a measurement of the bot and of the maintainers who received its comments. The post explains how a hit is counted; it does not describe how the denominator for recall is built ^[30].

Greptile has the highest precision in the group at 80.3 and a recall of 50.4, a gap of 29.9 points ^[12][22]. When it comments, developers usually act on it, and it leaves more issues unreported than the other leaders. Which of those two failures is cheaper depends on your gate. If a human triages every bot comment before the author sees it, recall is the scarce thing; if the bot posts straight to the pull request and the team has learned to skim it, precision is.

For the ordering to transfer, your reviewers would have to act on bot comments at roughly the rate the open source maintainers in this corpus did, and your pull requests would have to carry a similar mix of defects ^[20]. The post is direct about the first half: "Your closed codebase, compliance rules, and review culture are not that sample." ^[19]

Copilot's 63.9 rests on 767 pull requests, about 4.8 percent of the 16,017 the leaderboard has scored ^[23]. Qodo's 58.9 rests on a sample nearly three times as large ^[29]. Copilot's row has the most room to move as the corpus grows.

Set that against the ranking CodeAnt AI published. Its own 2026 roundup rates CodeAnt AI best overall, in a list it wrote, and its advertising carries a "300K-PR" figure that counts volume and not accuracy ^[2]. On the open leaderboard the tool scores 72.3 precision and 38.2 recall for an F1 of 50.0, the lowest of the seven rows the post lists ^[15][24].