Go back

Commission hesitant to join US evaluation system

The European Commission is being invited to join Star Metrics, a prototype US federal government system to track and monitor the impact of research funding. But Commission officials are said to be hesitant about participating.

US research agencies, led by the National Science Foundation, are piloting Star Metrics. Eighty-five research institutes have volunteered to participate in the programme, which aims to provide a systematic, comprehensive tracking of public research spending and its outcomes. All the data is collated electronically and automatically, requiring no extra work on the part of researchers, says NSF’s Julia Lane, who helped to design the system. It monitors the allocation of grants and jobs, and plans are afoot to cover other research outputs, such as citations and patents.

Lane says Japan has already launched Star Metrics activities and that Europe should be on board, too. “This is happening with or without the EU,” she told Research Europe, but the US is keen to share resources with Europe, in particular expertise in evaluation methods. Now is also a good time to join because Star Metrics is still in a prototype form and could be tailored to Europe’s needs, Lane says. The EU-US collaboration could begin with a Star Metrics pilot including several EU countries, she suggests, before a potential large-scale roll-out in time for the beginning of Framework 7’s follow-up programme Horizon 2020 in 2014.

A proposed “EU-US roadmap to measuring the results of investments in science” was presented at a European Parliament seminar in Brussels on 18 October, following a workshop held in Bellagio, Italy, in June. The roadmap consists of informal recommendations to “align efforts” and develop common tools.

The Commission has already been in discussions with the US officials on Star Metrics [see RE 318, p1 via link below], but hasn’t committed to using it to improve its own monitoring. Neville Reeve, a policy adviser at DG research who spoke at the Parliament seminar, said the Commission was “very aware” of the scheme’s potential. “A system similar to Star Metrics could in the longer run improve the quality of evidence and make [project] reporting much simpler,” he said.

The Commission will present its plans for evaluation and monitoring as part of its Horizon 2020 proposal at the end of November.