Jo Johnson chose the wrong man to turn the tide
What was universities minister Jo Johnson thinking when he commissioned eminent economist Nicholas Stern to review the Research Excellence Framework?
It’s no secret that ministers are looking for ways to reduce the REF’s £246 million bill—the bulk of which is spent by universities on preparation. But if they wanted to endorse a bigger role for metrics, and therefore a smaller role for peer review, they clearly chose the wrong man.
Instead, in a long awaited report, published last week, Stern and his team of academic advisers have retained their confidence in human intellect as the primary judge of scholarship. They sensibly do not rule out a role for metrics, but as a support for peer-review panels and not as an alternative, much as the previous The Metric Tide report from a group chaired by science policy expert James Wilsdon had recommended.
As we and others have been saying for months, even a cursory reading of Stern’s CV should have told the minister that he would not supply the answer Johnson was looking for. Stern’s experience in the economics of global development has taught him much about basing decisions on flaky data, and he wasn’t about to change his stripes.
The exercise was otherwise a model consultation. Those who submitted responses asked for a method to be found to decouple outputs from individuals, and the Stern team duly complied. At the same time the team recommended that all staff be submitted to the REF to eliminate the stigma for those researchers left out of the present system.
Research managers remain concerned that this could bring additional administrative burdens. Anticipating this, Stern’s team has said that institutions can reduce outputs per individual from four to two on average. It was thought that he might recommend a sampling system, but he decided against it on the grounds of cost and accuracy.
Smaller, less research-intensive universities also remain concerned as they have tended to be more selective in their REF submissions. They will now come under pressure, rightly, to ensure that there is a greater breadth of high-quality research across their institutions. As long as they continue to submit a similar number of top-rated outputs, as they have been doing, they are unlikely to experience loss of quality-related funding.
As a practitioner in the study and use of metrics in macroeconomics, Stern knows all about how indices can be gamed, and it was interesting to see what his report would recommend in this respect. His call for an end to “portability” of outputs is designed to subvert the pre-REF transfer market. The extent to which this works in practice remains an open question. As we report in this related article, younger researchers are viewing such a move as career limiting in an already hypercompetitive profession.
These are, of course, recommendations and it’s a moot point that the Department for Business, Energy and Industrial Strategy has not immediately endorsed the report. The action now moves to the funding councils to consult on the practical details. It is likely that Johnson and his team will apply pressure to include more of what they want. Stern and Wilsdon will know that the skirmish might be over but the war is not yet won.
This article also appeared in Research Fortnight