By day anesthesiologist John Carlisle works in a Torquay, UK hospital. In his spare time, he finds statistical errors in clinical trial research results. Carlisle has for the past decade become a clinical research detective in an intellectual “Dragnet” or hunt for truth: scrutinizing the published results of thousands of clinical trials in a serious, systematic and a specified manner so as to ensure those that either make a mistake or worse—correct their work. He has built a small team along the way and over time thanks to their hard work hundreds of papers have been retracted and corrected—due to either mistake (best case) or even misconduct (not the best case). The investigations have exposed “fakers” and ended careers.
TrialSite News offers a link to the story of John Carlisle, authored by David Adam and published in Nature. Carlisle has undoubtedly made some enemies along the way. How could he not? In this rushed, the winner-take-all world of today—where “publish or perish” takes on a new meaning, the Carlisle clinical Dragnet can catch perhaps relatively well-intentioned researchers feeling “under the gun to produce” to “true posers” that should be permanently banned from practice. Along the way some critique that at times the Dragnet may adversely impact well-intentioned researchers that made innocent mistakes.
Why the Dragnet? The Patients
Carlisle only cares about one thing and that is to protect patients. As with any true professional the energy, spark or driving force behind his work is intellectual curiosity and not an obsessive fervor to catch researchers in wrongdoing.
Gatekeepers of Science?
In the world of clinical research, academic and professional journals, as well as a research institution, should have the established stage-gates, reviews and scrutiny to ensure that the overwhelming amount of research results rest on sound, fundamentals. Unfortunately, based on Carlisle’s efforts, there are gaps as many an error is caught by the Carlisle research Dragnet and not the internal reviewers within the journals and institutions themselves. And as Adam points out in the Nature article—“In medical trials, the kind that Carlisle focuses on, that can be a matter of life and death.”
Caught in the Research Dragnet
Nature offers a link to a list of offenders in the Retraction Database: a database with lists of clinical researchers with the most retractions. An unofficial list, they include some with outrageously high numbers of retractions including Yoshitaka Fujii with 183 total retractions! Data can be searched and reviewed in this database.
For those that have the intellectual curiosity or professional interest, Retraction Watch is dedicated to reporting on retractions of scientific papers. The blog was launched in 2010 and is produced by science writers Ivan Oransky (VP, Editorial Medscape) and Adam Marcus (editor of Gastroenterology & Endoscopy News). The duo was motivated to launch the blog to further the interests of transparency in clinical research. They observed that retractions of clinical research papers, for example, generally are not announced and the reasons for the retractions are not publicized.
John Carlisle is listed, along with others, and is recognized for the significant work to expose the statistical anomalies of Yoshitaka who topes their leaderboard with 183 retractions not to mention his project to scrutinize more than 5,000 clinical trials.
Although the academic medical journals, scientific publications, and research institutions maintain time-tested, well-established systems and processes for the identification of academic or professional mistakes or worse, for whatever reason there are plenty of mistakes and even data manipulation that slips by reviewers employed by the most esteemed of academic journals for example. Could it be that funders, journals and many in the scientific community give a relatively low priority to such reviews? a dedicated group of professionals has set up a small, predominantly volunteer network of intellectual scrutinizers seeking to work for the cause of transparency in scientific and clinical research. They do this work for free—on their own time. We are thankful for such dedication and commitment.