The final set of RAE results is with us. The RAE is no more; on to the REF. But how much of a transition is there going to be from RAE to REF?
I suggest that the change from a single grade in RAE2001 to a three-part profile is as important as the future transition to the REF. The latter may well see greater use of metrics, directly and indirectly, but the provision of the profiles gives a much richer set of information than we’ve had before. (The profiles, published today, show the cumulative result for each submission. In January, institutions will receive the sub-profiles–for output, environment and esteem–that combine to make the cumulative result. The sub-profiles and the submissions will be generally available in Spring next year).
As before, we’ll be able to compare the input information (publications, postgraduate students per FTE and research income per FTE), once the data are released. Now, however, we’ll also be able to see more directly what the panels thought of those data and the other elements of the submission.
It’s probable that some submissions will have good ratings of their outputs, but that their environment can be seen to have pulled down their final rating. How might this additional information be used? Will it lead to greater attention on or support for departments to deliver on specific aspects of performance?
The profiles also introduce interesting opportunities for the funding councils, in terms of the funding mechanism that they choose to adopt. It goes without saying that there will be differential funding across the profile, but will that differential be the same for all subjects, as it has been in the past?
For example, if a subject was seen, nationally, to be internationally strong (that is, lots of 2*, 3* and 4* research), would it be reasonable not to fund 1* research? Equally, if a subject were weak, would it be reasonable to fund 1* and 2* at a higher rate, in order to try to support the development of the subject? Perhaps more controversially, if this is about funding institutions rather than subjects, per se, should the same approach be taken per institution?
A related issue is the allocation of the total funding across the subject areas. The approach differs between funding councils, but it needs to be based on the weighted combination of quality and volume in a subject, taking account of the relative cost of research, so that subjects get funded appropriately.
I wonder how much more informed we are now about the relative costs, which some believe are in need of updating.
A thought on presentational matters. It’s primarily the sector itself that gets uptight about the RAE and consequent league tables. But others do take note of the results. Nationally, we need to ensure that the transition from a seven point, discrete scale to a 0-4 continuum is adequately explained. Previously, ratings of 5 (with or without a multitude of stars) were “good”. Now, a grade point average (I’m not promoting them, but they will be used) of 2.5 might be “good” in one subject, even though quantitatively it is half the value, and 2.8 in another. How will we ensure that the presentation of the results to external audiences is adequately explained? What will be the new currency in adverts and promotional material? I imagine that it could be proportions or volumes of staff assessed as being internationally recognised (that is, 2* and above) or internationally excellent (3* and above). Whatever we use, we need to ensure that the change in rating scale does not disadvantage the UK’s external reputation.
The RAE has been criticised because of its burden and its behavioural effects. I’m yet to be convinced that the REF will significantly reduce burden, partly because the sector does not seem to like simplicity. Also, it will have behavioural effects; all systems do, even more so if there’s money or career prospects attached.
However, I am a believer in the RAE and the effects it has had on institutions taking more seriously their responsibilities towards understanding and managing of their research activity. This is not necessarily about being more directive towards researchers’ activity, but about recognising what needs to be in place to achieve excellence and impact, in the widest sense.
Finally, a reminiscence on my part. My first RAE was in 1996, when I managed the process for the University of Glasgow. The results also came out just before Christmas, and the day they were released happened to coincide with the University Court’s annual dinner. Having arrived back from the funding council bearing the results and an initial analysis done in the car, I met with the Principal and Vice-Principals, all in their dinner jackets and black ties. Somewhat surreal!
Ian Carter is Director of Research and Enterprise at the University of Sussex, and chairs the Association of Research Managers and Administrators.