Constructive advice on failed proposals is hugely valuable to early-career researchers, says Gemma Derrick
Discontent with how UK academia is governed is most obvious at the macro level. The past month has seen strikes over issues of overwork, pay, inequalities in research culture, and the future of academia. There has been increased social media chatter around decisions to leave academia, prompted by unworkable conditions, increased stress and strain on mental health.
But academics’ most frequent and significant interaction with the research system and its inequalities comes at an individual level, through peer review of draft publications and grant proposals. Repeated experience of applying for funding and failing, as most researchers do, contributes to the growing sense of malaise.
This is especially true for early career researchers seeking a place in a system where the number of PhD holders far outweighs the number of available jobs in research. Pursuing a future in academia means reaching milestones that show research independence. Here, the ability to navigate peer review is vital.
Is peer review obsolete?
Evidence of inefficiency, bias and arbitrariness in peer review has led some critics to question its role in a modern, transparent and accountable research system. There is an increasing interest in alternatives such as predictive indicators for success and lotteries.
But these methods bring biases and inefficiencies of their own. Better than rushing to replace peer review would be to understand how it can be made to work better.
Over the past two years, my colleagues and I have been working with the Wellcome Trust to study its grant decision-making processes for early career researchers. Using a combination of performance analysis, interviews and linguistic coding of reviewer reports, we have found the type of peer review that best supports a future career in research.
Unlike most previous studies, we focused on applicants who miss out. This reveals more about what works and what doesn’t in peer review than focusing on success stories. As we see it, reviewers have two jobs: to assist decision-making and provide feedback to applicants. Review should be about participation and development, not just assessment—both evaluating proposals and keeping applicants’ future in mind.
Currently, too many reviewers focus on the first function and overlook the second. One in our study branded an applicant who had authored two Nature papers within four years of completing their PhD as “insufficient”. Another suggested an applicant should only be considered if they moved to another country over 3,000 kilometres away.
Such unfair comments show an emphasis on separating success from failure, and on making judgments based on assumptions rather than offering advice based on the contents of the proposal. Forgetting that there are individuals, and sometimes vulnerable individuals, behind every application can lead to reviewing that is inappropriate and potentially harmful to young researchers.
Enhancing lives
Instead, reviewer feedback should aim to be actionable, targeted and fair. Peer review can and should be less about dividing winners from losers and more about giving the feedback and inspiration that academics need to make further applications and build their careers.
Good feedback sends applicants a positive signal, regardless of the funding outcome. It helps improve future applications towards future success.
In our study, most unsuccessful applicants went on to seek funding elsewhere. But those who received actionable and fair feedback were more than twice as likely to reapply than those who did not. Their training, effort and talent are less likely to leave academia.
Peer review that balances its two functions, seeking to help applicants and improve proposals, as well as providing a decision, is particularly important for early career researchers. They are less likely to have access to the resources and expertise needed to improve their application, and have less experience of previous funding applications.
To better serve applicants and the system as a whole, peer review should be less focused on decision-making and recognise its role in the development of researchers. This reimagining requires reviewers to realise that they are also mentors and provide feedback that helps the applicant as well as helping the decision-making process. It also means focusing less on replacing peer review with some alternative, and more on recognising and realising the potential of the existing process.
Implementing these changes is not about creating more work for already overstretched reviewers. It is more about changing perceptions of, and approaches to, peer review, to ensure the feedback it provides adds as much value as possible to the development of the next generation of researchers.
Gemma Derrick is an associate professor in the School of Education, University of Bristol
This article also appeared in Research Fortnight