At ICML, authors ranking their own papers beat reviewer scores at spotting future citations

The elicitation is the part worth copying. Authors asked for an absolute quality score will inflate it, so the experiment never asked for one. It asked which of an author's own submissions is stronger, a comparison in which nobody can place every paper first, and which the paper grounds in game-theoretic reasoning about when truthful ranking is an author's best available move ^[9]. It is also an easier question than assigning a number ^[9].

Divide the submissions covered by the researchers who ranked them and the ratio is 1.93 ^[4]. Since eligibility required multiple submissions, every participant ranked at least two, so arithmetic forces at least 92 ranking slots onto submissions that somebody else also ranked ^[4]. That matters for the tail result, where 77 percent of the papers above 150 citations were top-ranked by at least one author ^[3]. "At least one" is a softer bar than agreement among co-authors, and the text as published does not give the agreement rate ^[6].

The incumbent it beat is not a precise instrument either. In a NeurIPS 2021 experiment, roughly half the accepted papers would have been rejected under a second independent review ^[13]. Outperforming review scores as a citation predictor ^[7] is therefore a low bar cleared, not a high one; the interesting property is that it was cleared by a question authors answer about work they already know, rather than by more reviewer hours. The robustness checks help here: the associations held after accounting for final decisions, review-score ranges, preprint posting dates and self-citations ^[8].

The axis is narrower than the framing suggests. Peer review is charged with establishing factual correctness as well as identifying scientific potential ^[14], and citation counts test neither correctness nor the reasons a paper was rejected. The paper concedes citations are an imperfect measure of impact ^[15]. So the honest statement is that authors predict one downstream proxy better than reviewers do, on a proxy reviewers were never asked to forecast.

Volume is what makes a free signal worth arguing about. ICML submissions went from 1,676 in 2017 to 12,107 in 2025, a factor of 7.2 ^[10][1]; NeurIPS went from 3,240 to 21,575 across the same years, a factor of 6.7 ^[11][2]. That is 33,682 submissions between two conferences in one year ^[5], against a reviewer pool that did not grow in step, which is why graduate and undergraduate reviewers without prior publications at these venues, and language models, are now doing the work ^[12].

Read against that, self-rankings are a prior, not a verdict. The defensible use is ordering attention: which papers get a third reviewer, which rejections get an area chair's second look, which of an author's submissions absorbs the scarce senior reviewer. The moment a ranking carries weight in the decision, the incentive that made it informative changes, and the experiment measured the version where it did not.