Image: Klaus Vedfelt, via Getty Images
University survey reveals dissatisfaction with way metrics are applied to evaluate research
Over two-thirds of University of Cape Town academics have told a survey that the way research impact is currently evaluated pushes researchers away from non-scholarly activities like teaching, and disadvantages the global south and soft sciences.
The survey of 119 UCT academic and research staff across disciplines did not set out to evaluate commonly used metrics, such as citations or journal impact factors. Instead, it explored challenges with their use in distributing funding and making appointments.
The survey—conducted by Andiswa Mfengu and Jaya Raju, and published in the July-August edition of the South African Journal of Science—found that 72.3 per cent of respondents felt the use of research metrics impacted researchers’ behaviour, for example by prioritising publishing over mentoring students.
A researcher interviewed for the study stated that “university, institution and government don’t care about what we do practically as researchers. They care only about publications”.
Of the respondents, 73.1 per cent also said the current use of impact indicators also undervalues the contribution of researchers from the global south and unfairly favours hard sciences. “The global south tends to be excluded from the knowledge production ecosystem and therefore scholarship from the global south is partly invisible and inaccessible,” Mfengu and Raju write.
Broader impact
Mfengu and Raju, both based at UCT’s Department of Knowledge and Information Stewardship, argue that their UCT case study shows how current bibliometric indicators “fail to capture” the broader impact of research, such as mentoring, data sharing, public engagement or giving opportunities to underrepresented groups.
“The use of metric indicators tends to exert pressure on researchers to change their behaviour and research agendas to conform to these norms at the expense of locally relevant scholarship,” Mfengu and Raju write.
The authors call for more “responsible” use of research metrics, and efforts to complement quantitative metrics with others measuring quality. “Moreover, there needs to be a concerted national conversation on research assessment reform in South Africa among higher education leaders, funders and policymakers,” they add.