Understand the real reasons reproducibility reform fails
A decade ago, the US National Institute of Neurological Disorders and Stroke convened a workshop on how to improve the rigour of preclinical research. Its recommendations were surprisingly straightforward: scientists should mask (or ‘blind’) their studies; randomize; estimate appropriate sample sizes; and specify rules for data handling (S. C. Landis et al. Nature 490, 187–191; 2012). Ten years on, many preclinical scientists still do not take these basic steps.
Ask most advocates of rigorous science why this is, and they will answer with two words: perverse incentives. Scientists are rewarded for getting things published, not for getting things right, and so they tend to favour speed and ease over robustness. But as an ethnographer, this explanation has never sat well with me. I’ve spent more than 15 years studying biomedical research cultures, and scientists’ behaviours are rarely so transactional. So I decided to knock on a few doors at my institution, the University of Wisconsin–Madison, to ask researchers who work with animals why they were using the methods they were.
They explained that their decisions were based not on publication pressures, but on maintaining the integrity of experiments and respecting facility routines. For example, techniques to mask which treatment groups animals are in risk causing misidentification or cross-infection.