The rules for assessing submissions to the Research Excellence Framework will vary greatly between research fields, documents published by the Higher Education Funding Council for England suggest.
The four main assessment panels published draft criteria and working methods for their 36 sub-panels on 29 July. Each has decided on different approaches to the use of citation data as a measure of research output, as well as what they consider to be research “impact”.
HEFCE has already confirmed that overall impact will be weighted to account for 20 per cent of the total assessment, while research outputs will account for 65 per cent and research environment 15 per cent.
The documents show that while panel B, which covers the physical sciences, will make use of citation data only in sub-panels 7, 8, 9 and 11—Earth and environmental sciences, chemistry, physics and computer science—all of panel A research, which broadly covers the life sciences, will use such data.
However the panel will only use citation data “as a minor component to inform peer-review judgements”, says the draft guidance.
Meanwhile no panel D sub-panels—which cover areas including languages, English literature, history, classics and art—will make any use of citation data or journal rankings.
The majority of sub-panels reporting to panel C, which covers fields such as architecture, law, politics, sociology, education and sports science, will not use citation data when assessing submissions. None will use journal impact factors.
However, two panel C sub-panels—17, covering geography, environmental studies and archaeology, and 18, covering economics and econometrics—will use citations as part of their assessments. In the case of sub-panel 17, this will apply to some areas of physical geography and environmental studies but not to archaeology or human geography.
Both sub-panels will be “alert to any potential bias that might arise from using citations data,” says HEFCE.
The criteria also vary in the range of outputs that can be included in assessments.
Physical scientists, for example, will be able to submit patents, book chapters and computer algorithms and software as evidence of research output, alongside peer-reviewed publications.
Life scientists meanwhile will be only be able to include outputs such as textbooks, databases or abstracts “exceptionally”, where they embody original research.
Although impact is defined broadly by HEFCE as including a variety of effects across the economy and society, panels also differ on whether effects on teaching can be included.
Within panel B, research impact can include actions that have an effect on teaching or students where they extend significantly beyond the submitting institution. However panel C will not accept such examples in its impact assessments.
The draft criteria also differ on whether a co-authored output may be listed against more than one member of staff returned within the same submission, and whether staff will have to include a description of their personal contribution.
HEFCE has opened a consultation on the assessment documents, which is open until 5 October. The final criteria and working methods will be published in early 2012.
HEFCE will launch a pilot programme for submissions by December 2012, with all universities making submissions by 29 November 2013. Assessment will take place during 2014 for allocation of research funding from 2015-16.
Guidelines for universities, published earlier this month, outlined the information university units will have to include within submissions.
These include information on staff, details of publications and other “assessable outputs”, the approach to getting “impact” from research and previous examples, as well as data on research PhDs, research income for the period and a description of the research environment.