Cooperative Research Centres and early career researchers lead criticisms of excellence framework
Australia’s national framework for the evaluation of university research has been described as a poor investment of academics’ time.
The Australian Research Council is holding a review of the Excellence in Research for Australia standards, as well as engagement and impact assessment guidelines. ERA assessments have been carried out every three years for the past decade.
In its submission, the Cooperative Research Centres Association said that for the CRCs, “ERA scores contribute very little to the choice of research organisations participating in a bid, its operations, or the quality or impact of outcomes”.
There is a “significant” time lag in the ratings, the CRCs said. “An assessment of the past quality or past impact is of very limited value…In our experience, industry parties do not choose their university collaboration partners on the basis of the ERA or engagement and impact scores.”
Furthermore, the institution-level ratings are “not fit for purpose” as CRCs are “much more interested in individual researchers or their lab group than a department or university”.
“High-performing individuals exist in poorly rated departments and low-performing individuals exist in highly rated departments,” the association said.
It added that the engagement and impact assessment guidelines had helped motivate researchers to work more closely with industry, but that changes to university block grants had made more impact in that area.
There is a tendency to “game” both sets of rankings, the association said, and the cost of data collection is very high.
“We know that academics are considerably occupied by data collection when it is happening, with the attention of senior researchers in particular often being significantly diverted, and we know that there are people dedicated to the task. It would not surprise us if it was the equivalent of 20 [full-time-equivalent staff] for each university. At that rate, each ERA exercise costs the nation something in the order of A$100 million,” the submission said, adding that the return on investment for this work was not sufficient.
“Our view is that most of the gains of the ERA were realised with its first iteration. We can see no argument that justifies conducting the exercise more frequently than once per decade and, even then, great caution should be taken before proceeding,” the CRCs said.
‘Once-in-a-decade opportunity’
The Australian Academy of Science’s Early and Mid-Career Researcher Forum said in its submission that reformed ERA guidelines should include all research, not just that done at universities. It said the review was a “once-in-a-decade opportunity for the Australian Research Council to consider how ERA, engagement and impact operate, and whether they continue to be fit for purpose”.
The forum said it was “concerned that the current ERA processes fail to capture the contributions of early and mid-career researchers to research, and fail to capture the role of non-university research organisations and groups”. ERA processes are skewed towards more senior and less diverse researchers, and reformed guidelines should measure the retention and development of early and mid-career staff, it said.
The Group of Eight universities, which received around two-thirds of Australian Research Council grant funding in 2018, recommended changes to the ERA system. Chief executive Vicki Thomson asked the review for streamlined data collection methods, a longer assessment cycle of five years and a reporting system that included volume as well as quality of research.
A spokesperson for the research council said the review had been completed and that its recommendations would be released “during the second quarter of 2021”.