Posted by Britt Hermes on February 28, 2016 Real medicine is built on high quality data. That’s one lesson I learned through my rehabilitation as a former naturopath. I can assure you that this principle does not apply to naturopathy or any of the other types of alternative medicine, which are mostly based on anachronistic traditions, magical thinking, and poor academic integrity. I now know that real medicine incorporates any therapy scientifically demonstrated to be effective and safe, regardless of origin. Therapies that fail to demonstrate effectiveness remain alternative for good reason. I made a sharp turn in my career…
Real medicine is built on high quality data. That’s one lesson I learned through my rehabilitation as a former naturopath. I can assure you that this principle does not apply to naturopathy or any of the other types of alternative medicine, which are mostly based on anachronistic traditions, magical thinking, and poor academic integrity. I now know that real medicine incorporates any therapy scientifically demonstrated to be effective and safe, regardless of origin. Therapies that fail to demonstrate effectiveness remain alternative for good reason.
I made a sharp turn in my career when I realized this, but I was recently surprised to learn that a failure to respect the data can sometimes also characterize medicine. I’m not trying to fool anyone with an appeal to hypocrisy (often used by alternative medicine zealots to argue that their beliefs are somehow valid simply because medicine has problems). But I do want to draw attention to a problem with medical research, which has the potential to undermine science-based medicine and take us down the dark path of misinformed medicine — medicine based on incomplete data.
Most people probably don’t realize that a majority of clinical trials do not report their findings.[1–3] About half of trials conducted before 2007, before it was legally required to register and report clinical trials, have never been published.[4, 5] Even since stronger legal mandates were enacted, most clinical trials still are not registered or reported.[1–3] As a result, there is a huge amount of clinical data unknown to the medical community and not available to clinicians or patients.
This effectively means that medical decisions can be based on incomplete information, not only diminishing the effectiveness of treatments but also eroding the perceived plausibility of medical interventions over time. This is what Sense About Science’s AllTrials campaign is all about: ensuring that all clinical trials register and report results in a publicly-available database. I see it as an obvious and straightforward means to improve medicine.
A brief history of trial registration laws and regulations
The ClinicalTrials.gov registry — a database of clinical trials — was created in 2000 by the National Institutes of Health (NIH) National Library of Medicine with help from the Federal Drug Administration. It was created as a response to the 1997 U.S. Federal law requiring the registration of all clinical trials involving human participants in an open-access database.[6] This was an important legislative step intended to disseminate information about clinical trials to the public and researchers.
Over the past eight years, there have been various policy improvements in reaction to poor trial registration compliance. Most notably, the Food and Drug Administration Amendments Act (FDAAA) Section 801, implemented in 2007, requires sponsors of applicable clinical trials to register and then report results at ClinicalTrials.gov. The trials affected by Section 801 include all non-phase I trials of drugs, medical devices, or biologics that
- began after September 27, 2007 or, for those started earlier, were still underway by December 26, 2007;
- have a research site located in the U.S., or
- run under “an Investigational New Drug application or an Investigational Device exemption.”[2, 3]
The FDAAA stipulated that all trial results must be reported within 12 months of collecting data for the primary research outcome. If a study ends early, any results must be reported within one year of the termination date (some exceptions were made for patent protection).[2] In parallel, the Declaration of Helsinki was amended by the World Medical Association General Assembly to include prospective registration prior to subject recruitment and mandated disclosure of study results to the public.[6]
In January 2015, the National Cancer Institute (NIH) issued a new policy echoing the FDAAA’s requirements to report results within 12 months of the trial’s primary completion date. Importantly, this new policy has been implemented into the conditions for receiving funding.[6]
What is being reported? Not much.
In March 2015, The New England Journal of Medicine published a review by Anderson et al. stating that only 13.4% of clinical trials either completed or terminated between January 1, 2008 and August 31, 2012 reported results within the mandated 12-month time period; only 38.3% of trials had reported results within 60 months (n=13,327).[2] Based on these measly percentages, it is safe to say that many researchers keep their clinical trial results under wraps.
In the midst of rampant underreporting across the board, there is an interesting difference between studies funded by the pharmaceutical industry and those funded by government and academic sources. Based on a random sample of 205 clinical trials out of the 13,327 reviewed by Anderson et al., about 80% of pharmaceutical industry trials reported results or had a legal reason for delay within the 5-year reporting period. In contrast, trials funded by government and academic sources did far worse. Only about 50% of NIH funded trials reported results or had legal reason not to, and other government and academic institutions reported at a rate of about 45%.[2] Overall, the pharmaceutical industry is doing a better job following the law than clinical trials funded by other sources.[1]
It seems that ethical and legal obligations are not drivers for following the rules of reporting.
So what is going on? Despite obligations to report results of clinical trials, researchers clearly fail to do so in a timely manner, if ever. This inaction may be explained by two factors: too much effort and a lack of incentive to publish negative results, and a lack of regulatory oversight.
No carrot: reporting is too much work and there’s no incentive to do it
Reporting results demands effort and when results are negative and unlikely to be published in medical journals, there is little academic incentive to prepare reports for public consumption. Ideally, researchers would budget the time and money needed to report results into their grant proposals, and funding organizations would force researchers to comply, as the NIH is mandated to do, at least on paper.
In a statement to STATnews by a Stanford University communications official, Lisa Lapin, expedient reporting comes down to time and money: Stanford researchers do not have enough time or resources to register and report clinical trial results, which results in 95% of their trial results being reported late or not at all since it became legally required to do so.[1]
This effect seems to spawn from the well-known and well-studied tilt towards positive results known as publication bias, which is in turn often attributed to the cutthroat nature of academia. Scientists are judged on research performance, including number of publications, where their contributions are being published, and the overall impact of their research.[7] In such an environment, positive results seem more interesting and exciting than negative results and are rewarded in what acts as a positive feedback loop.
Outside of academia, clinical trials funded by pharmaceutical companies tend to have more results reported than trials funded by the NIH or other scholarly organizations. This differential makes sense to me given how the pharmaceutical sector is subject to great scrutiny. “Big Pharma” is constantly targeted for alleged misconduct. It is reasonable to think that a pharmaceutical or medical device company would not want to open up vulnerabilities by withholding clinical data. Recently, I happened to meet the high-ranking employee, in charge of this task at an international drug company a few months ago, who reiterated this sentiment. She stated specifically that, amongst her other duties, she keeps track of trial results needing to be reported.
No stick: lack of regulatory oversight and no enforcement of penalties
The other reason for non-industry researchers not reporting data is a lack of regulatory oversight. Penalties for not reporting results have been set by the FDAAA, but they are not enforced. According to Stuart Buck at HealthAffairs.org, the FDAAA law could result in a total collection of penalties over $60 million dollars per day if the law were enforced![8] This figure is based on the $10,000-per-day fine that could be imposed on the 6,599 trials (registered from 2008 to 2012) that failed to report findings on time.
Federal authorities have never enforced the legal requirement to report these results.[8] This lack of enforcement certainly sends the message that the United States does not value the scientific practice of reporting all clinical trial results under the ethical guidelines that scientists would probably claim to respect. The FDA Deputy Commissioner, Dr. Robert Califf, has suggested he will begin to enforce the FDAAA, but so far the FDA has not levied penalties.[1, 9] Curiously, Dr. Califf requested his name be removed from three publications outlining changes for clinical trial management.[10]
The problems with not reporting clinical data
Unpublished research is harmful to medicine and its moral standing. A failure to report trial results is a breach of the ethical obligation to human subjects set forth in the Declaration of Helsinki. Since clinical trial research depends on human volunteers, it is absolutely essential that researchers uphold their end of the deal by making results available, if not for the volunteers, but for others, who may benefit from the findings. Patient advocate AnneMarie Ciccarella reflects this sentiment on the Sense about Science USA website:
We provided our bodies, our tissue samples, our data. I’ve heard the same sentiment expressed many times from patients in clinical trials, ‘This may not help me, but it may help another person.’ It’s time to honor that sentiment.
Undisclosed clinical trial results may not hold answers for better treatment options but may be concealing the ineffectiveness or dangers of treatments. Every patient, physician, and researcher should have access to this vital information that is largely withheld from surveillance. Tragically, an ongoing news story proves to be a powerful example of why no trial should be exempt from registering trial results and basic information, including the identity of the substance under investigation.
On January 17, 2015, a participant of a Phase I clinical trial died after developing brain damage caused by an experimental drug under way in France by the company Biotrial, and five other subjects have been hospitalized for symptoms consistent with brain hemorrhaging. The drug, developed by a Portuguese firm intended to treat a variety of symptoms including anxiety and motor dysfunction associated with Parkinson’s disease as well as chronic pain caused by cancer, was being tested in humans for the first time.[4] As of the writing of this post, neither French authorities nor Biotrial have released specific information regarding the identity of the molecule in the trial.
For now, scientists can only speculate about what substance was being tested in this trial beyond the vague information that the drug is a fatty acid amine hydrolase (FAAH) inhibitor. The truth is that Biotrial has not broken French law or international standards regulating clinical trials.[5] While adverse events in Phase I clinical trials are generally thought to be very rare, we actually do not have data available to independently verify this widely held notion.[5] For now, we can only hope that the French authorities will act in the best interests of these patients and install safeguards against future harm in first-in-human studies.
One obvious step away from problems with trial secrecy is to mandate and enforce trial registration and data sharing of trials at any phase, which includes enough information for the trial to be fully understood pharmacologically and therefore reproducible. The International Committee of Medical Journal Editors (ICMJE) proposed in an editorial published concurrently in 14 major medical journals that authors seeking to publish in these journals must share anonymous individual patient data associated with the results of any clinical trial.[7] The editors state that such patient data is required to reproduce clinical trials and is a tenet of good and ethical scientific practice.
While new policies such as this one are well intentioned, an IPD sharing policy may not support the bigger, and more pressing issue of unavailable trial summary results. Dr. Ben Goldacre, a British physician, science writer, and co-founder of the AllTrials campaign for clinical transparency, summarizes the mismatch between the ICMJE’s policy regarding IPD and scientific need thusly:
Overall, therefore, while greater support for IPD sharing is definitely great news, it’s very odd that all this resource and effort is now going into IPD sharing, when IPD sharing is expensive and not in demand, with complex governance and access control issues, whereas summary results sharing presents no privacy issues, and has been highly in demand, for many decades, but with multiple ongoing barriers to access, and frustratingly slow progress.
The proposals for IPD set forth by the International Committee of Medical Journal Editors are necessary and important. Undoubtedly, policies that uphold ethical obligations in research should be implemented. However, we have a more urgent ethical responsibility. Before venturing into the complex and costly task of sharing IPD, the scientific community needs to address the problem of trial secrecy being perpetuated through lax legal requirements and failure to enforce clinical trial registration, including the drug agent under investigation, followed by timely trial summary results. It seems to be time for an overhaul.
A campaign with solutions
AllTrials is an international campaign demanding responsible and ethical scientific research through the registration and reporting of all past and present clinical trials. Included in its mission is a mandate that it be impossible to receive funding or to sell a product without proving registration of clinical trials first.
Spearheaded by Ben Goldacre, the AllTrials campaign was launched by Sense About Science, BMJ, the Centre for Evidence-Based Medicine, the Cochrane Collaboration, the James Lind Initiative, and PLOS in January 2013 in the UK. In July 2015, the US AllTrials campaign was launched collectively by Sense about Science USA, Dartmouth’s Geisel School of Medicine, and the Dartmouth Institute for Health Policy & Clinical Practice.
The US AllTrials campaign calls for clinical trials to be registered with ClinicalTrials.gov. In total, over 634 medical organizations, including major journals, patient-advocacy groups and professional societies have joined the AllTrials campaign. This is a much needed force for change that if successful would help provide higher transparency for research conducted on human subjects. This is an important moral issue.
If all researchers publicly share negative results in a comprehensive manner (like they are legally and ethically mandated to do) we may start to cultivate a culture of research that embraces humility and honesty. In my opinion, not only will this open modus operandi help researchers resist temptations to cheat, it will also help the public learn to trust the scientific process. Without awareness of what has already been researched, scientists may be wastefully repeating clinical trials or asking questions for which answers could be found in unpublished data. In worst-case scenarios, clinical trials are protected from scrutiny even in the event of participants dying and subsequent investigations impeded by poorly implemented and enforced laws that should be protecting the integrity of patients and promoting the highest ethical standards of scientific research.
Most skeptics would probably agree that one of the most frustrating aspects of discussing science-based medicine with people who do not trust science is trying to convey that, for the most part, science is reliable and that the endeavor is honest. If scientists consistently uphold the ethical duties by reporting all results, the public may be more willing to engage in the scientific enterprise, and researchers will benefit from a greater awareness of existing scientific data to further improve their inquiries and contextualize their findings within a more complete prior corpus of data.
For information about how you can help, visit and AllTrials.net.
References
- Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM: Compliance with Results Reporting at ClinicalTrials.gov. N Engl J Med 2015, 372:1031–1039.
- Pansieri C, Pandolfini C, Bonati M: The evolution in registration of clinical trials: a chronicle of the historical calls and current initiatives promoting transparency. Eur J Clin Pharmacol 2015, 71:1159–1164.
- Patients endangered as law is ignored [http://www.statnews.com/2015/12/13/clinical-trials-investigation/]
- Butler D, Callaway E: Scientists in the dark after French clinical trial proves fatal. Nature 2016, 529:263–264.
- French clinical trial debacle highlights weak reporting laws [http://www.statnews.com/2016/01/20/french-clinical-trial-debacle/]
- US Federal Government Efforts to Improve Clinical Trial Transparency with Expanded Trial Registries and Open Data Sharing. AMA J Ethic 2015, 17:1152–1159.
- Taichman DB, Backus J, Baethge C, Bauchner H, de Leeuw PW, Drazen JM, Fletcher J, Frizelle FA, Groves T, Haileamlak A, James A, Laine C, Peiperl L, Pinborg A, Sahni P, Wu S: Sharing Clinical Trial Data — A Proposal from the International Committee of Medical Journal Editors. N Engl J Med 2016:160120050013003.
Britt Marie Hermes is a naturopathy apostate: she practiced as a licensed naturopathic doctor in the United States for about three years, but then left the profession to pursue a science-based career. She is now a Master’s of Science student in Medical Life Sciences at the University of Kiel. Her research interests include inflammatory and genetic diseases, like psoriasis and Crohn’s. She lives in Kiel, Germany, with her husband, who is a doctoral candidate in archaeology, and their two dogs. She recently started the blog Naturopathic Diaries: Confessions of a Former Naturopath.
Source: Reporting results from clinical trials is vital for science-based medicine : Science Based Medicine
