Research on Research Institute
Browse

<b>Applicants as Reviewers: Evaluating the Risks, Benefits, and Potential of Distributed Peer Review for Grant Funding Allocations</b><b>  </b>(RoRI Working Paper No. 17)

Download (1.82 MB)
Version 2 2025-08-27, 10:48
Version 1 2025-08-27, 10:47
preprint
posted on 2025-08-27, 10:48 authored by Research on Research Institute (RoRI)Research on Research Institute (RoRI), Anna ButtersAnna Butters, Melanie Benson MarshallMelanie Benson Marshall, Stephen PinfieldStephen Pinfield, Tom StaffordTom Stafford, Alexander Bondarenko, Barbara Neubauer, Robert Nuske, Pierre Schwidlinski, Hanna Denecke
<p dir="ltr"><b>This paper presents a mixed-methods evaluation of an experiment with distributed peer review (DPR) carried out by the Volkswagen Foundation.</b></p><p dir="ltr">DPR is an alternative funding evaluation model in which applicants also act as reviewers for other proposals in the same call. The study compares DPR with conventional panel review, examining efficiency, fairness, and robustness of funding decisions, as well as participants’ experiences and concerns.</p><p dir="ltr">Findings highlight both the advantages of DPR - including scalability, diversity of feedback, and reduced time to decision - and potential risks such as gaming, workload, and limits on reviewer expertise.</p><p dir="ltr">The paper concludes with strategic recommendations for funders considering the implementation of DPR.</p>

Funding

RoRI’s second phase (2023–2027) is funded by an international consortium of partners, including: Australian Research Council (ARC); Canadian Institutes of Health Research (CIHR); Digital Science; Dutch Research Council (NWO); Gordon and Betty Moore Foundation [Grant number GBMF12312; DOI 10.37807/GBMF12312]; King Baudouin Foundation; La Caixa Foundation; Leiden University; Luxembourg National Research Fund (FNR); Michael Smith Health Research BC; National Research Foundation of South Africa; Novo Nordisk Foundation [Grant number NNF23SA0083996]; Research England (part of UK Research and Innovation); Social Sciences and Humanities Research Council of Canada (SSHRC); Swiss National Science Foundation (SNSF); University College London (UCL); Volkswagen Foundation; and Wellcome Trust [Grant number 228086/Z/23/Z].

History

Related Materials

Usage metrics

    Research on Research Institute

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC