Science That Is Not Transparent Is Bad Science: Richard Gray On the Citation of Retracted Articles

May 24, 2018 Danielle Chilvers

I recently interviewed Richard Gray, Editor of the Journal of Psychiatric and Mental Health Nursing and Professor of Clinical Nursing Practice at Latrobe University, Melbourne, who spotted a retracted Clinical Trial which, prior to retraction, had been included and cited in two Review articles. An Editor’s Note has since been published alongside the Review articles to inform readers of the situation, and below, Richard has summed up the reasons why this was important, along with advice on how to maintain the integrity of the published literature.

Richard Gray.jpg

Q. Thanks for taking the time to speak with us Richard. So citations were made to a retracted article which occurred before the article was retracted. You spotted this, investigated, and concluded that the retracted article had no substantial bearing on the conclusions of the two review articles. Why is it important to follow up on this?

A. Thank you for asking me. Your question – I guess – is prompted by an exploratory study our group did that sought to test if systematic reviews ever include retracted papers. The work matters because, in a clinical discipline such as nursing, practice should be informed by the best available evidence, which generally comes in the form of a systematic review. If a review includes a retracted paper then it is possible that conclusions and recommendations for practice may be unsound. In our study we identified 37 systematic reviews that included a retracted clinical trial. Two of these were published in Wiley journals (the Journal of Clinical Nursing and the Journal of Advanced Nursing). The focus of both reviews was on mobile phone interventions to enhance medication adherence. To be clear, the authors of these systematic review articles had probably not made a mistake, the retraction occurred after the review had been published. And you are right, the conclusions of the two reviews in Wiley journals were not substantially affected by the retraction of an included study. However, we did find a number of examples where review conclusions were markedly affected by the removal of a retracted trial. This included one review where recalculating the meta-analysis (minus the retracted trial) changed a non-significant effect to one that was significant. Ultimately, we argue readers should have all the facts about a review to enable them to make an informed judgement about the quality and reliability of the research they are using to inform their clinical practice.

Q. Why all the fuss about transparency?

A. Rigor, replicability and retraction are fundamental to the scientific method. The first two require researchers to be detailed and meticulous in the reporting of their research. If a serious error is made in a study or there is misconduct the paper reporting the results should be retracted (removed from the evidence base). Many readers will be familiar with the efforts of drug companies making well known antidepressants to either distort the reporting of trials to their commercial advantage (the infamous study 329) or avoid publishing inconvenient studies altogether. Although their motivations are different, researchers in academic settings are not immune. Taken together these problems create the various forms of publication bias. Science that is not transparent is bad science. The work of our group on the inclusion of retracted trials in systematic reviews might seem a bit bookish, but I think our work extends thinking about transparent reporting of science in a new and important way.

Q: What’s your advice to Editors, Reviewers and Authors? How can they be more vigilant about this sort of thing?

A. I think journal editors, reviewers and authors need to be more vigilant about the issue of retraction and systematic reviews. Certainly, authors and reviewers need to check carefully the status of manuscripts included in reviews. This is a tedious, boring task, but one vital to upholding the integrity of the review. We suspect, but have no definitive evidence, that review authors come across duplicate publications in the course of conducting their research that they do not declare (in their paper or to publishing editors). We have pondered if authors of systematic reviews should have an “ethics” section in their manuscripts, stating how they will respond if they identify duplicate publications or other research misconduct in the course of their reviews.

It is not clear who is responsible for alerting journal editors that a citation in a systematic review they published has been retracted. The Cochrane handbook has something to say on the matter, suggesting that review authors are responsible for monitoring the status of included trials. A nice idea, but I suspect that few authors ever do this (I am happy to be proved wrong). In my opinion it is incumbent upon the editor who retracts a paper to alert the authors citing the study that the paper has been retracted. Hard pressed editors may resist having to take on yet another burdensome administrative task. But think about the problem for a few minutes: if they don’t do the work, who will?

Thanks Richard, for your thoughts and advice to other Editors on such an important topic. We at Wiley commend you for taking the time to assist us in alerting readers.

Links to the Editor’s Notes can be found here and here.

Image Credit: Richard Gray



Previous Article
Plagiarism : What's the Truth?
Plagiarism : What's the Truth?

Plagiarism: What's the Truth?

Next Article
Upholding Research Integrity and Publishing Ethics
Upholding Research Integrity and Publishing Ethics

Chris Graf, Director of Research Integrity & Publishing Ethics at Wiley, discusses some of the major concer...