Australian Research Council’s advice for government says assessment should provide maximum value for minimum effort
The Australian Research Council’s blueprint for a national research assessment system has been revealed in a submission to the Australian Universities Accord panel.
The ARC says the new system should include a national research data infrastructure and follow five principles: maximum value for minimum effort; balanced benefits; connection with other systems; diversity; and trust and relevance. It should be “up to date” and of a standard that “sets it apart from other sources”.
The advice for the government, which dates from December 2022 and is framed as a “transition plan” as the country moves away from the scrapped Excellence in Research for Australia and Engagement and Impact assessment processes, was kept confidential until last month. It was made public in September as part of the council’s response to the draft recommendations of the Universities Accord advisory group, which is informing a new agreement between higher education institutions and the government.
The transition plan includes greater use of automated data about research outputs but warns that these data will need to be “curated”, possibly with the help of artificial intelligence. A high priority will be the use of persistent identifiers so that published research can be more easily identified.
Work underway
The Accord response also reveals that the ARC has started preliminary work to support the new assessment system and is “investigating smarter harvesting of data for evaluation of the research system, including for non-traditional research outputs, data linking using persistent identifiers, data curation and auto-coding techniques, and a deep dive into Indigenous studies to pilot some of these new approaches”.
“The ARC has begun expanding the ideas outlined in the transition plan and engaging with stakeholders to better understand where there would be ‘early wins’ and appetite for larger reform,” the submission says. It is also carrying out “a stocktake” of available data on research outputs and developing a study of the recently created Indigenous research codes to identify where the work is having an impact.
The council urged the Accord panel, led by former New South Wales chief scientist Mary O’Kane, to support the principles of its transition plan. The panel released its interim report in July. Its final report to education minister Jason Clare is due in December.
On 4 October, O’Kane released an update noting that 300 submissions had been made on the draft recommendations. She said she had also met the heads of parallel reviews into early childhood and school education systems, and that “structured consultation” with key stakeholders was still going on.
Transition plan development
The transition plan was developed at the request of Clare in a 2022 letter of expectations. He said he wanted a transition plan that could “be considered for implementation in 2024-25”.
While the results of a recent review of the ARC have already been released, the question of a new research assessment system was deferred to allow for the Universities Accord process to play out.
The ARC’s response to the Accord draft says that “recommendations on a new model are to be provided to the minister for education for consideration alongside the Accord panel’s final report”.
The ARC said the transition plan was written in consultation with universities in “a modern data-driven approach informed by expert review”. It added that it had continued to evolve its thinking since then as it consulted others across the sector, but “the key principles and approaches to commencing a consultation process with the sector, all outlined in the transition plan, remain highly relevant”.