Go back

Working together towards reproducibililty


Oxford initiative aims to link people and disciplines, say Laura Fortunato and Dorothy Bishop

Efforts to enhance the reproducibility of UK research are growing rapidly. The UK Reproducibility Network, the national umbrella organisation coordinating these efforts, is just over a year old, but it already has 47 local networks at institutions across the UK, and has recruited 29 stakeholders, including funders, journals and learned societies.

Since December, 11 universities (Aberdeen, Bristol, Edinburgh, Keele, Newcastle, Oxford Brookes, Reading, the Royal Veterinary College, Sheffield, Surrey and University College London) have affiliated to the UKRN. This means committing to appoint an academic lead for research improvement within the senior management team, responsible for championing open research and reproducibility.

At our institution, the University of Oxford, a host of people and initiatives working on reproducibility have sprung up over the past five years, initially without any co-ordination. These included seminar series, journal clubs, training opportunities and attempts to modify incentives by changing criteria for hiring and promotion. Those involved included early-career researchers impatient for change, as well as more senior academics, and the library and research services staff responsible for implementing initiatives on issues such as open access.

Gradually these groups became aware of one another and recognised the need for co-ordinated action to pool expertise, identify gaps, and avoid duplication of effort. And so Reproducible Research Oxford was born.

RROx (pronounced ‘rocks’) is the largest local network of UKRN. Within Oxford, we aim to help the university pursue a coordinated approach to reproducibility that extends across all disciplines. Nationally and internationally, we aim to share best practice and ideas, highlighting the collaborative ethos of efforts towards open research.

Until recently, RROx was run largely by volunteers giving their own time and expertise. As its activities have grown, this has become unsustainable. In December, with support from internal university funds, RROx appointed its first coordinator, Malika Ihle. Her brief is to integrate different strands of work across all disciplines.

To mark this development, on Monday 13 January RROx held an official launch event at St John’s College, Oxford.

The day was a chance both to take stock and to see what remains to be done. One message, for example, was that researchers often overlook how much library and IT services have to offer on reproducibility and open research. Another focus was on early-career researchers, who gave a series of short, inspirational presentations describing what they have already achieved, concluding with a wish list of changes.

To date, scientists have been at the forefront of initiatives around research reproducibility and open scholarship. It is important to broaden the debate, so that practices from science are not imposed on fields where they might not work. RROx aims to start a dialogue with people working in the humanities, for example, to identify practices with multidisciplinary relevance and uncover subject-specific needs.

The launch event was an opportunity to set a course for RROx, via discussions within small groups mixing people from different subject areas and constituencies. The discussions of existing activities, future needs, priorities and potential unintended consequences provided a wealth of material to shape RROx’s agenda.

We also heard from some external speakers. Claire Fraser, senior higher education policy adviser at Research England, was able to dispel some myths about how funders shape researcher behaviour. She noted that Research England had endorsed open access and responsible use of metrics in evaluation, and that the Research Excellence Framework was increasingly moving towards rewarding open, reproducible scholarship.

Stephen Curry of Imperial College London appeared in his capacity as chair of the steering committee for Dora, the San Francisco Declaration on Research Assessment. He noted a pleasing growth in the number of higher education institutions that had signed up, but warned that there is still a need for firm policies to ensure adherence.

Oxford signed Dora in 2018, and has recently released principles for the responsible use of research metrics. RROx will promote these across the university.

The final speaker was Brian Nosek from the University of Virginia, co-founder and director of the Center for Open Science. He has long worked to develop infrastructure, in the form of the Open Science Framework, that makes it easy for academics to engage in transparent, open research. It was encouraging to see data documenting the rapid growth of open practices in his own discipline, psychology. Across fields, change is in the air, making it the perfect moment to launch RROx.

Laura Fortunato is associate professor of evolutionary anthropology at the University of Oxford. Dorothy Bishop is professor of developmental neuropsychology at the University of Oxford. Both are members of the steering group of Reproducible Research Oxford.

A version of this article also appeared in Research Fortnight