Skip to content

Leadership1 publisher3 min readPublished

NIH's plan to hide peer-review scores swaps a contestable number for a 25-point band

A Request for Information posted on a Friday evening would replace numerical impact scores with three broad bands. Comments close October 13.

The Board Room · Leadership desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

Photograph accompanying NIH's plan to hide peer-review scores swaps a contestable number for a 25-point band
Photo: forbes.com

What happened

  • NIH posted an announcement on August 14 titled "Request for Information (RFI) on Proposed Changes to Reporting Outcomes from NIH Peer Review," which proposes concealing from applicants the scores their grant applications receive from scientific peer review, and gives until October 13 to submit comments via a link in the RFI.
  • Under the proposal, applicants would see only which of three overall impact categories their application fell into: "Most competitive" (based on the top 25% of final overall impact scores), "Competitive" (based on the 26-50% of final overall impact scores), or "Not discussed" (scores greater than 50%, or ND).
  • Traditionally an NIH grant application is reviewed by a committee of external scientific peers, each of whom assigns a numerical score; the applicant's final score is a composite of those scores, and in most cases that composite determines whether the application gets funded, assuming enough funding is available.
  • Applicants could historically compare their score against published thresholds called paylines for a given NIH Institute, Center or Office (ICO), which indicated whether they were in the ballpark of funding and how far away they were, information that could guide revision for resubmission or a decision to try a different approach.
  • "External" review means outside NIH and outside the federal government; this was done to prevent politicians and anyone inside government from having undue influence on what grants get awarded to whom, and to ensure reviewers had some degree of relevant scientific expertise.

Compiled by The Board RoomSomething wrong?How this is made

Why it matters

The National Institutes of Health posted a Request for Information on August 14 titled "Request for Information (RFI) on Proposed Changes to Reporting Outcomes from NIH Peer Review," proposing that grant applicants no longer be shown the numerical scores their applications receive in scientific peer review [1]. Instead of a composite score, an applicant would see one of three categories: "Most competitive," drawn from the top 25% of final overall impact scores, "Competitive," covering the 26th to 50th percentile, or "Not discussed," meaning scores above the 50th percentile [2].

That matters because the composite score has been the operative number in the system. Applications have traditionally been reviewed by a committee of external scientific peers, each assigning a numerical score, with the composite in most cases determining whether the application is funded when funding is available [3]. Applicants could then compare their score to published paylines for a given Institute, Center or Office, which told them whether they were close and, if not, how far off, and therefore whether to revise and resubmit or abandon the approach [4]. According to the Forbes account, external review meant outside NIH and outside the federal government, structured that way both to keep politicians and government insiders from exerting undue influence over awards and to ensure reviewers had relevant expertise [5]. Reviewer lists have been made available to the applicant and the public [6].

The loss here is not the score itself but the gap. A band spanning 25 percentiles cannot be compared with a payline expressed as a threshold, so an application sitting just outside the funding line and one sitting well outside it become indistinguishable to the principal investigator [7]. That is the same information a research office uses to decide where to put bridge funding, which resubmissions to back, and which programs to wind down. If your internal rules key off proximity to a payline, they will need a new trigger, because the input will not arrive.

The second consequence is discretionary room. Forbes columnist Bruce Y. Lee frames the central question as how far the change, if enacted, moves NIH funding decisions away from science and scientists and closer to the U.S. President and his political appointees [8]. He notes the announcement went up late on a Friday, before the weekend, when few people are watching [9], and attributes the proposal to NIH under the Trump administration [10]. That is a single-sourced characterisation, but the structural point stands on the mechanics: a number is contestable in a way a category is not, and an applicant told only that they were "Not discussed" has nothing specific to argue with.

Comments are open until October 13, roughly 60 days from the posting [1][11]. Two things are worth tracking in whatever emerges. First, whether NIH says anything about the future of published paylines and public reviewer lists, since bands without paylines remove the last external benchmark [4][6]. Second, whether institutions with large NIH portfolios file comments on the record, because the comment file is the only formal leverage available before the policy is set [1]. In the meantime, treat internal scientific review as your own scoring system and keep the numbers, since it may soon be the only quantified read your investigators get.

Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories