The way we do peer review is highly problematic, says Remco Heesen, philosopher of science at the University of Groningen. Talking to Laura Molenaar, he proposes a radical alternative.
Science doesn’t stop when the research is done. After running experiments and interpreting findings, the great majority of scientists go on to publish their work in a peer-reviewed journal. It is then distributed to colleagues, often behind a paywall, and used further down the line as evidence of good scientific performance in job applications and funding bids.
Remco Heesen, a philosopher of science at the University of Groningen and the University of Western Australia, is highly critical of this process. As a scientist, he has first-hand experience with peer review and has grown increasingly concerned about its power.
“It’s hard to avoid,” he says. “Where you publish has major consequences for your career in terms of funding and getting a position.”
Heesen uses mathematical methods from decision theory and economics to model how scientists share information. “There’s no doubt that any scientist uses strategic reasoning: when choosing what and where to publish, and for how long to work on a project before publication,” he says. “These components exist aside from considerations of intrinsic scientific value.”
His research has made Heesen attentive to a number of problems in the peer review system. “One problem has to do with the period of time between writing the paper and publication. During that time, the results aren’t shared with the scientific community, so they can’t be benefited from.”
But the trouble doesn’t stop there. When a paper is sent to a journal for review, Heesen says, the editor selects peer reviewers who take up the task of judging whether the paper deserves publication—normally, one to three reviewers are appointed for this task. “But that’s really not enough to form a reliable judgment of the quality of a paper,” Heesen says.
Research on peer review scores has shown that whether a paper is published is more a question of luck than scientific value, he says. “Given two reviewers, the probability that one recommends accepting a paper and the other rejecting it is nearly as large as the probability they agree.”
A third issue is money—a debate that has spawned calls around the world for a boost to open-access publishing, where papers are made available for free.
“Right now, the papers published by scientists who are often paid with public funds disappear behind a paywall,” Heesen says. “Then, publicly funded libraries need to pay again in order to get access to journals.” He says that this process excludes people not affiliated with a university and its library from accessing research results.
“It costs about US$5,000 (€4,500) to publish a paper. Even with open-access journals, you’ll only save about half of that cost. If you publish in an online archive, however, the costs will amount to about $15 per paper.”
From his research, Heesen has developed an alternative proposal to traditional publication. It would involve scientists publishing immediately, with reviews being written as more and more scientists read the paper.
“You can think of it like the movie critics website Rotten Tomatoes,” he says. “Other scientists could comment below the publication or on a separate blog. They might even add a numerical score to their review.”
Heesen says this would have several advantages. “First of all, everything is made openly accessible.” That would mean that everyone can immediately read and benefit from the results. “Furthermore, it is much easier to see which papers are well received, because of the use of a score system.”
He’s not the only one who has called for an alternative to the existing peer review system. In 2016, Jean-Sébastien Caux, professor of theoretical condensed-matter physics at the University of Amsterdam, launched the website SciPost, where peer reviewers see an article in advance, after which their comments are published online next to the article.
Heesen expects that his system would make reviewing cheaper and raise the quality of the papers on offer.
“This alternative system makes life easier for the reviewers as well,” he says. Reviewers can make a few comments and score the paper immediately, rather than having to write a detailed exegesis of why a paper should be accepted or rejected by a journal. “I think having lively discussions online will be stimulating for the reviewers,” Heesen adds.
A further advantage is that the system would be less reliant on the judgement of reviewers, as their opinion would not determine which papers were published. This could make the scientific publication process fairer to scientists without established networks, and prevent peer review bias.
“Peers can publish their opinion, but everyone can see what they write. Right now, only the editor and possibly the author ever read what a reviewer has written.”
So what role would journals play in Heesen’s alternative publication system? His answer is clear. “As far as I’m concerned, we can do without those for-profit publishers completely,” he says.