Moves against emphasis on culture are mistaken, say Stephen Curry, Elizabeth Gadd and James Wilsdon
When the proposals for the next Research Excellence Framework were published in June, reactions were broadly positive.
In the past five years, the acute problems in research cultures have received steadily more attention from university leaders, policymakers and funders. REF 2028’s increased attention to people, culture and environment offered a route to tackling these concerns, rebalancing incentives towards the collective and collaborative aspects of a healthy, dynamic and fair research system.
Yet as summer edged into autumn, in some quarters that confidence seems to have collapsed. Critics are circling the REF 2028 proposals, hoping to dilute or scrap them altogether, in favour of the status quo.
This would be a mistake, subverting efforts to improve UK research culture through the uniquely influential mechanism of the REF and effectively kicking the can to the framework after next in 2035.
Three-pronged attack
Arguments against the proposed reforms fall into three camps. Some protest that reducing the weighting towards research outputs from 60 per cent to 50 per cent of the exercise risks compromising its focus on the ‘excellence’ of UK research. This view hinges on the contention that research culture is an input, or enabler, not an output, so should not be factored into assessment.
A second line of attack is that the reforms sound nice in principle, but are technically complicated or unworkable: how do you define research culture, never mind measure it?
The final camp is trying to drag the REF into the culture wars, portraying any attempt to fold people and culture into assessment as part of the creep of ‘wokeism’ or a threat to academic freedom.
Each critique raises important issues that need open, honest and evidence-informed discussion. To instead hope that surreptitious lobbying or scaremongering will throttle the proposals before new methods can be elaborated and the wider research community consulted runs counter to the spirit of the process.
Swimming with the tide
In December 2022, our Harnessing the Metric Tide review set out the rationale for these reforms. In June, the Future Research Assessment Programme’s heavyweight International Advisory Group did the same, in fact recommending that REF 2028 give equal weighting to outputs, impacts, and environment and culture—a significantly more radical model than the one adopted.
Such findings draw on more than a decade of international dialogue, which has identified the many harmful unintended effects–such as hyper-competition, loss of creativity and research integrity, inefficiency, short-termism and bias—that result from evaluation processes too focused on published scholarly outputs.
Initiatives such as the Declaration on Research Assessment, Latin American Forum in Scientific Evaluation and Coalition on Advancing Research Assessment, to name a few, agree that definitions and indicators of excellence should embrace not only outputs, but also the processes, environment and outcomes of research.
Over the past five years, UK funders and institutions have steadily aligned with this international consensus. UK Research and Innovation has endorsed the government’s R&D People and Culture Strategy, while in its 2021 report, Realising our Potential, the Russell Group calls a healthy research culture “crucial to attracting and retaining staff, ensuring a research-led recovery and building the UK’s reputation as a global leader in research”.
Deja vu
This makes it disappointing that, given an opportunity to put their money where their mouth is, some have opted for a hasty retreat. Such back-pedalling reflects both a tightening in the wider funding climate and the realpolitik of research assessment, which has still not internalised critiques of a narrow view of excellence.
One puzzling aspect of the pushback is that people and culture already figure prominently in the REF’s assessment of research environments, which made up 15 per cent of the 2021 exercise.
The latest proposals represent an evolution, not a revolution, giving greater emphasis to the accepted view that without a culture that lets people thrive, there can be very little excellent research. Adjusting the weightings is a logical step—and one taken before with the introduction of impact in REF 2014.
Then, as now, many argued loudly that the move would be damaging, daft or technically impossible. But the government and funding councils held firm, and 10 years on, the inclusion of impacts is regarded as one of the REF’s better features.
Any new element of research assessment may bring unforeseen consequences. But like research cultures, impacts take many forms, showing that the REF can handle concepts that are hard to define. And, as happened with impact, changes to the assessment of research culture would surely be consulted on and piloted.
Proceed with care
All sides can agree that we need to proceed with care—codifying research cultures raises genuinely tricky questions. However, the means by which the REF plans to achieve this—using the Scope framework, which encourages evaluators to look for unintended consequences and gauge costs and benefits; co-creating the evaluation with those who will be evaluated; and combining qualitative and quantitative approaches—are established principles of responsible research assessment.
Safeguarding the vitality, qualities and impacts of UK research means getting this right. Abandoning these proposals now, or putting them off to 2035, would suggest that all the commitments to improving research cultures by ministers, funders, vice-chancellors and researchers themselves are just so much lip-service.
It is time for our world-leading research institutions to demonstrate genuine leadership in how we measure, manage and evaluate the things that matter most in research.
Stephen Curry is a consul and professor of structural biology at Imperial College London, chair of the Declaration on Research Assessment, and director of strategy at the Research on Research Institute.
Elizabeth Gadd is research culture and quality lead at Loughborough University, chair of the research evaluation group of the International Network of Research Management Societies, and a vice-chair of the Coalition on Advancing Research Assessment.
James Wilsdon is professor of research policy in the Department of Science, Technology, Engineering and Public Policy at University College London, and executive director of the Research on Research Institute.
A version of this article also appeared in Research Fortnight