‘Publish or perish’ culture blamed for reproducibility crisis
Survey of more than 1,600 biomedical researchers also flagged small sample sizes and cherry-picking of data as leading causes of reproducibility problems
Laurie Udesky
Nearly three-quarters of biomedical researchers think there is a reproducibility crisis in science, according to a survey published in November. The leading cause cited for that crisis was “pressure to publish”.
The study, reported in PLoS Biology, surveyed the authors of articles that were published in the year from 1 October 2020 in any of 400 randomly selected biomedical journals. The 1,630 respondents represented more than 80 countries. The majority were male (59%) and were faculty members or primary investigators (72%), and 42% worked in the United States, Canada or the United Kingdom.
Sixty-two per cent of respondents said that pressure to publish “always” or “very often” contributes to irreproducibility, the survey found.
That “really speaks to the fact that it’s about the culture in the research ecosystem that’s proliferating this problem” — a problem of valuing quantity over quality, says Kelly Cobey, a social psychologist at the University of Ottawa Heart Institute and lead author of the study.
A documented problem
The problem of irreproducibility in science has been documented in other fields and has been well known for decades, says Cobey. The study’s aim, she adds, was to build on a 2016 Nature survey in which more than 70% of 1,576 respondents said they had trouble reproducing other scientists’ research.
1,500 scientists lift the lid on reproducibility
Other perceived causes of research irreproducibility that were flagged in the current study as “always” or “very often” contributing included small sample size (55% of respondents), studies being completed but not reported (54%), flawed statistical analyses (50%) and cherry-picking of results (47%).
“I think this is spot on with what many of us have been talking about or suspecting in the last 20 years — this rise in all kinds of errors,” says Elisabeth Bik, a microbiologist and science sleuth based in the San Francisco Bay Area, California, whose doggedness has led to more than 1,300 retractions so far. “Some of it might be sloppy, and some of it looks like it was misconduct.” Bik, who began her science-integrity work after discovering that one of her own papers had been plagiarized, agrees that the pressure to publish is the main culprit.
Ivan Oransky, co-founder of the Retraction Watch website, says that since the site began tracking retractions in 2018, he and his colleagues have logged some 54,000 in their database. Two-thirds of these were attributed to misconduct — that is, falsification, fabrication and plagiarism.
To fix the problem, he says, upstream changes will be needed in the culture that assesses value according to the number of citations. “Until we acknowledge that publish or perish is the natural consequence of the rankings obsession, we’re not going to fix anything.”
Systemic changes will also be required, says Marcus Munafò, a biological psychologist at the University of Bristol, UK, and the co-founder of the UK Reproducibility Network.
One of the network’s major focuses is training researchers in open research practices — “sharing their data, sharing their codes, sharing their study materials”, Munafò explains. “Collaborating in that way just reduces friction in the system to everyone’s benefit.”
But changing the system more widely will necessitate an expansion of the model currently used by reproducibility networks, Munafò says — including broader outreach to researchers, funders, publishers and academic institutions — so that “policies that are being scoped out by funders, for example, can be informed by those grass-roots communities”.
Cobey agrees. Initiatives such as the UK Reproducibility Network have been good at raising awareness and developing training and tools, she says. “But the reality is that the systematic retraining and re-incentivizing around things that will support reproducible research hasn’t happened and it won’t happen until there’s more coordinated action amongst different stakeholders in the research ecosystem.”
“The consequence of not doing this is public harm,” she adds.
LEGGI ANCHE
Cobey, K. D. et al. PLoS Biol. 22, e3002870 (2024).
