Gengyan Tang, MA1Hao Cai, MS2
Question Do paper mill articles contaminate systematic reviews in the life sciences, and what are their implications for the integrity of evidence synthesis?
Findings In a cross-sectional study of 200 000 systematic reviews published between 2013 and 2024, 0.15% incorporated retracted paper mill articles into the evidence synthesis, with an increase over time. Oncology was the most affected field, and a total of 124 citations occurred after retraction, including 13 occurring more than 500 days after the retraction date.
Meaning These findings highlight the growing risk of citation contamination in systematic reviews and underscore the need for enhanced screening measures, systematic correction of contaminated reviews, and automated detection tools to protect the integrity of evidence synthesis.
Importance Systematic reviews are the criterion standard for evidence synthesis in the life sciences, yet their reliability and integrity are threatened by citation contamination from fabricated publications produced by paper mills. Despite growing awareness, the extent and implications of this issue remain unclear.
Objectives To analyze the prevalence, characteristics, affected subject areas, and citation patterns of retracted paper mill articles cited in systematic reviews.
Design, Setting, and Participants This cross-sectional study analyzed systematic reviews published between 2013 and 2024, indexed in Web of Science (WoS). References were matched against the Retraction Watch dataset, and full texts were reviewed to identify retracted paper mill articles incorporated into the evidence synthesis.
Main Outcomes and Measures The study assessed (1) contamination prevalence, defined as the proportion of systematic reviews incorporating retracted paper mill articles into the evidence synthesis; (2) geographic distribution of citing authors according to institutional affiliations; (3) citation timing and trends, including the time lag between incorporation and article retraction; (4) affected research areas, categorized by WoS subject classifications; and (5) citation patterns, including highly contaminated reviews (≥3 incorporations of retracted articles).
Results Of the total of 200 000 systematic reviews, 299 incorporated at least 1 retracted paper mill article into the evidence synthesis (contamination rate, 0.15%). Among them, 256 (85.6%) included a single retracted article, and 43 (14.4%) included multiple such articles. Of 1802 author affiliations associated with the contaminated reviews, 660 (36.6%) were from institutions in China. Of 385 total citations, 124 (32.2%) occurred after retraction, including 13 occurring more than 500 days after the retraction date. Oncology was the most affected field (48 of 299 [16.1%]). Five reviews each included 5 or more retracted articles, all published in journals under questionable publishers.
Conclusions and Relevance In this cross-sectional study of life sciences systematic reviews, contamination remained low but increased over time, posing a risk to research integrity. Continued citation of retracted articles, even after retraction, highlights the need for rigorous screening practices. Correcting contaminated reviews and developing automated detection tools are essential to preserving the credibility of systematic reviews.
