CSR is committed to strengthening the peer review process. Learn about our commitment and relevant data.
Developing the Simplified Framework
For many years NIH heard frustrations from the external community about the complexity of the peer review process and increasing responsibilities regarding policy compliance. These issues have the potential effect of diverting the attention of reviewers away from their key task of evaluating the scientific and technical merit of grant applications. Additionally, NIH heard concerns that reputational bias has an undue influence on peer review outcomes.
To help address these problems, NIH developed revisions to the peer review process through the Simplified Framework for NIH Peer Review Criteria. In forming the simplified framework, NIH gathered input from multiple sources over a period of years.
In 2019 and 2020 the Center of Scientific Review (CSR) formed external working groups and published a 2022 Request for Information which received more than 800 responses (PDF) from individuals and scientific societies. Final recommendations were considered by the major extramural committees of the NIH that included leadership from across NIH Institutes and Centers, and resulted in the modifications of the final proposal presented at NIH’s Advisory Council to the Director in November and December of 2022.
Improving Fellowship Review
CSR led efforts to revise the fellowship application and review process. The goal was to help reviewers better evaluate the trainee’s potential and the quality of the scientific training plan without the undue influence of the reputation of either the sponsor or the institution. The revised framework was developed with input from the extramural community, including responses to a 2023 Request for Information (PDF). Through these changes, NIH aims to improve the chances that the most promising fellowship candidates will be consistently identified by scientific review panels.
Training to Strengthen the Integrity of Peer Review
CSR developed training to raise awareness of threats to review integrity, such as breaches of confidentiality and foreign influence. The training relies on real-life cases and provides strategies for reviewers to navigate and report such situations. A survey completed in 2024 (n = 22,838 respondents) indicated that the training is effective in helping reviewers maintain review integrity and encourages them to report integrity concerns for investigation.
Training to Mitigate Unfairness in Peer Review
CSR developed targeted training to mitigate the most common biases in the peer review process, such as reputational bias. The training includes interactive exercises and a narrated mock study section demonstrating techniques to intervene based on real examples. The training has been required of all NIH reviewers since 2024 (NOT-OD-23-156).
Reporting Avenues for Unfair Reviews
CSR has a reporting avenue for issues related to the fairness of the review process. The reporting channel is open to all – investigators, reviewers, and program staff.
See Something, Say Something
Report concerns about a compromised review by contacting your scientific review officer or send a message to [email protected], which will reach CSR senior leadership.
Report a Concern| Investigate | |
|
|
Every allegation is carefully investigated by CSR senior management. |
| Resolve | |
|
|
If we agree the review was flawed, CSR will re-review application in the same council round, if at all possible. If CSR does not agree, the official NIH appeals process remains available to all investigators. |
| Closure and Culture Change | |
|
Re-review? CSR Scientific Division Director discusses the issue with the reviewer, if appropriate. Actions might also include not inviting the reviewer to serve in the future. |