Understanding how research works doesn’t mean reinventing the wheel, says James Wilsdon
On 30 September, 150 research funders, policymakers and academics from 19 countries gathered at the Wellcome Collection in London for the launch of the Research on Research Institute. This consortium aims to champion transformative and translational research on research systems, cultures and decision-making.
With 28 talks in under eight hours, the meeting provided a tasting menu of some of the most creative and innovative work underway to understand the inner workings of research systems. Speakers outlined experiments with approaches to funding, evaluation and support to create more diverse and inclusive research cultures.
My highlights included: Stephen Curry of Imperial College London on progress towards redefining research excellence and leadership; Jude Fransman of The Open University on the need to shift the focus in global debates about research from effectiveness to equity; Molly King of Santa Clara University on gender differences in patterns of self-citation; Wilhelm Krull of the Volkswagen Foundation on an ongoing trial of partial randomisation in some grant processes; and Nature editor Magdalena Skipper on recent experiments with peer review.
The day ended with a reminder from Tom Kariuki, director of programmes at the African Academy of Sciences, of the importance of “research with”, not just “research on”, in advancing these agendas with international partners. Beyond the formal sessions, it was exciting to see the debates sparked at the meeting and online.
Building on a rich history
Over the next decade, policymakers, funders, universities, publishers and researchers will need to navigate changes in their operating environment. These include the continuing expansion of global science and a greater emphasis on interdisciplinarity, mission and challenge-directed research, open scholarship and open data, collaboration and team science, and the imperatives of diversity, inclusion, integrity and reproducibility.
Rich seams of theoretical and empirical work on these questions flow through several disciplines, dating back decades. They can be found in science and technology studies, innovation studies, scientometrics, policy studies, higher education studies, the history and philosophy of science, and library and information sciences.
More recently, heightened concern over the quality and reproducibility of research has prompted waves of self-criticism across the biomedical sciences, psychology and economics. Researchers in these disciplines have spearheaded the establishment of high-profile centres, networks and conferences dedicated to addressing these issues.
These activities can raise eyebrows among more established communities working on these topics, who see newer entrants’ claims to novelty as the product either of genuine ignorance or of a more disingenuous rebadging aimed at securing territory, funding and influence. To give a recent example, the Metascience 2019 conference, which took place at Stanford University from 5 to 8 September, claimed that “during this decade, we have witnessed the emergence of a new discipline called metascience, meta-research, or the science of science”.
At the meeting, many presenters acknowledged their intellectual debt or self-identification within more established fields. But the broader framing inevitably provoked some criticism, particularly at the annual meetings of science and technology studies and scientometrics, which took place in the same week in New Orleans and Rome, respectively. The closing plenary of the New Orleans meeting saw some pointed criticism directed towards those who were “Columbusing”—claiming to discover something that wasn’t new—related fields.
Changes and challenges
The Research on Research Institute, a partnership between the Wellcome Trust, the company Digital Science, the University of Sheffield and Leiden University in the Netherlands, will not treat research on research as a new discipline. Rather, we see it as a lens through which to organise, analyse and advance a breadth of disciplinary, theoretical and methodological approaches to questions about the governance, design, operation and evaluation of research systems.
At the same time, we need to be clear about the ways in which these questions, the contexts in which they are being asked and our capacity to answer them are changing. Data is part of the story: we now have more sophisticated indicators and analytic tools relating to many dimensions of research and researcher performance, outcomes and impacts, going well beyond standard citation and patent measures.
There is greater recognition of the value of integrating and combining methods to develop real-time intelligence on how research systems are performing and changing. These possibilities will expand as applications of data science in and on research systems are linked to machine learning and artificial intelligence.
Funders are beginning to use these advances to underpin decision-making. Some are going further still, creating internal teams to undertake such analysis and build decision-support tools. Examples include the Meta team at the Chan Zuckerberg Initiative and the Wellcome Data Labs. In the process, how funders support and evaluate research is starting to change.
Worldwide, the challenges of mobilising, scaling and aligning research efforts in pursuit of the Sustainable Development Goals show that robust, interdisciplinary and translational research on research is needed more than ever. The same goes in the UK, with the need to ground debates over prioritisation and balance (or any fresh attempts by Dominic Cummings to reorganise the UK research system).
We hope the Research on Research Institute can play a positive role in building this evidence base and strengthening the alliances needed to translate it into practice. And we warmly invite others to join us in this task.
James Wilsdon is director of the Research on Research Institute and Digital Science professor of research policy at the University of Sheffield (@jameswilsdon).
A version of this article also appeared in Research Europe