Publication databases still miss much of the world’s research, say André Brasil and Ludo Waltman
A couple of years ago, one of us conducted a study on the development dynamics of graduate education in a southern state of Brazil. The research was of little interest to international audiences, but relevant to local researchers and policymakers. Thus, it was written in Portuguese and published in a Brazilian journal not indexed in any international bibliographic database.
Consequently, the paper is nearly invisible in global university rankings and research analytics based on journal impact factors. However, unlike many other countries, Brazil has an evaluation system that credits local journals. Hence, the work benefited from the highest score a journal could receive in the country’s classification system.
Today, bibliographic databases, particularly Scopus, owned by Elsevier, and Web of Science, owned by Clarivate, provide data on scientific publications covering a substantial share of global research outputs. Nevertheless, in geographical and linguistic terms, their coverage is insufficient. (Research Professional News is an editorially independent part of Ex Libris, which is owned by Clarivate.)
For instance, researchers in Brazil publish about 100,000 articles every year, but an analysis by one of us in 2021 showed that Web of Science indexes only half of them. Moreover, while more than two-thirds of articles in the health, life and exact sciences are covered, for the social sciences and humanities, the figure is below 10 per cent. Most of the missing papers are in Portuguese.
A lot of research is of global relevance, and dissemination in international English-language journals makes perfect sense. However, much research deals with locally relevant topics, for which dissemination in local journals is often preferable, especially as many academics face language barriers that limit their access to research.
Regionally relevant publications are sometimes indexed in local bibliographic databases; in Brazil, these include SciELO, RedALyC and Latindex. But many of these publications are invisible in international databases, and so are missed by evaluations that draw on these sources.
There is growing awareness of the importance of locally relevant research. This is emphasised in the Leiden Manifesto for responsible research assessment, on which one of us is an author, and the Helsinki Initiative on Multilingualism in Scholarly Communication.
The San Francisco Declaration on Research Assessment sets out concrete steps to promote global inclusiveness in research assessment, and countries such as China are trying to move away from an over-reliance on English-language journals. Diamond open access publishing, where authors are not charged for publication, is often offered by journals with a national or regional focus.
But as long as bibliometric indicators and other research analytics are based on a partial view of the scholarly literature, taking into account only outputs in selected journals, assessment systems will push researchers towards topics these journals consider to be of interest. This undermines a globally inclusive research system.
Locally relevant research
To properly recognise locally relevant research, research analytics need to be globally inclusive. Therefore, publishers, infrastructure organisations, funders, policymakers and research evaluators need to set up infrastructures that provide globally inclusive data on research outputs, without excluding particular places, languages or topics.
Scopus and Web of Science have broadened their coverage of the global scholarly literature. However, their continuing reliance on restrictive content selection by experts excludes a lot of valuable research outputs.
Alternatives are emerging from a mixture of commercial and non-profit infrastructures that are trying to find ways to be globally inclusive, such as Crossref, Dimensions, DOAJ, OpenAlex, and OpenCitations. None is entirely successful yet, but they have made commendable steps along a challenging path.
Being involved in research evaluation ourselves, we know that readily available standardised information is essential to support robust evaluation processes. But if evaluation uses information from databases with an incomplete picture of the scholarly literature, there is a risk of discouraging certain types of research, particularly on locally relevant topics.
In addition, researchers should defend their right to publish on locally relevant topics in local journals, and to get recognition for doing so. A genuinely equitable research system can be realised, but only if we value research outputs regardless of geographical focus, language and publication venue.
André Brasil works for the CAPES Foundation, the Brazilian federal agency for quality assurance in higher education. Ludo Waltman is professor of quantitative science studies at the Centre for Science and Technology Studies, University of Leiden
This article also appeared in Research Europe