Go back

REF 2021: ‘So is that it? Certainly not’

Image: Research England

Early signs suggest research culture becoming more significant in future evaluations, says David Sweeney

Well, that was the week that was—I’m old enough to remember the TV show. Thanks to Research England’s REF director Kim Hackett and her wonderful team, results day went smoothly, and it seems that almost everything that could be written has been written.

There was good analysis in national and local papers, and the trade press and the Higher Education Policy Institute distinguished themselves by presenting their own analyses and soliciting excellent articles from informed and engaged commentators. Research Professional and Times Higher Education disarmed my criticism of rankings by drawing attention to different forms of league table.

Institution after institution highlighted their successes, hopefully without triggering any comment from the UK Statistics Authority. Webinars brought forth challenging questions and the former science and universities minister David Willetts—on whose watch the foundations of the Research Excellence Framework were laid—provided a magisterial summary and some kinds words, which I appreciated.

So is that it? Certainly not.

The REF provides an immense set of information, notably in the panel summaries and the huge set of case studies. These will be published over the next couple of months, with the main panel’s overview reports having already appeared on 18 May.

Big data

These data will allow consideration of the big-picture questions that can only be answered after synthesis and analysis of the additional material. In Research England, working with the other UK funding bodies, we are commissioning formal evaluations that will complement the work of academics, professional services staff and external commentators in drawing conclusions from a massive evidence base.

Public investment in research is increasing, and we hope private investment will follow. Collectively, all of us supporting the research system want to use that investment efficiently and effectively, and we want to be part of broader debate about structure and culture of the research and development ecosystem.

In considering how the REF has changed since 2014, and looking forward to the next assessment, there is considerable focus on granularity—researchers, research teams, research portfolios in institutions, right up to the UK’s ‘performance’ in what is a global collaborative, and competitive, enterprise.

The REF does not look at individuals—we have already destroyed our temporary store of personalised information—but at disciplinary portfolios. But there are, of course, names attached to research outputs, albeit very often many names, and the employment relationship is between individuals and their institution.

Nicholas Stern’s 2016 review of REF 2014 advised that the exercise should recognise the work of all those employed by their institution with a significant responsibility to do research. I think implementing that recommendation has given better information.

Similarly, partially decoupling the link with people, to allow a variable number of outputs for submitted staff has eased the pressures on individuals. We will look at further decoupling the link with individuals, moving away from a lens of ‘my research’ to better recognise that responsibility for setting research directions is shared between funders, institutions and researchers.

International effort

Looking ahead, the Future Research Assessment Programme is set to report in the autumn. Early consideration of our recently ended Frap consultation draws attention to the need to maintain the rigour of the exercise and to determine the nature of the next exercise promptly.

We have also received many comments about paying more attention to developing a better environment for research, with a special focus on issues around research culture. As yet, this interest has not crystallised in a way that would contribute to an assessment exercise and we need to broaden our engagement on this.

We have already sought a refresh from the Metric Tide group to update the 2015 report. Along with our International Advisory Group, we are working on responsible research assessment with the Global Research Council and will engage with a similar European initiative announced at the Paris Open Science conference earlier this year.

Thinking about research culture and environment issues, I admire the progress made in the pilot phase of the Research on Research Institute, which has built a collaborative consortium of 21 partners, including many funding agencies, from 14 countries.

All have a shared commitment to transformative and translational research on research systems, cultures and decision-making. The UK funding bodies and Frap will work with these initiatives, to contribute insights from our own work and ensure the UK draws on international best practice in building its contribution to the global research system.

There is lots to ponder, much to analyse and consider, and pressure to get on with it. So let’s do that together.

David Sweeney is executive chair of Research England

A version of this article appeared in Research Fortnight