Study Suggests Most Clinical Trials Are Scientifically Unjustified, Unethical

"Clinical trial" is a nice way of saying "human medical experiment." Experimenting on humans is ethical, so long as the people who volunteer give informed consent and receive a treatment that is thought to be medically beneficial.

That latter criterion makes some clinical trials ethically impossible. We could, for instance, prove definitively that vaccines do not cause autism by randomizing a group of children to receive vaccines and another group not to receive vaccines. But not vaccinating children is unethical, so this experiment could never be done.

A similar line of thinking explains why we can't feed people donuts every day to see if they develop diabetes. Though many people might willingly sign up for that clinical trial, it is unethical to design an experiment with the intention of causing a disease to develop. Furthermore, clinical trials that test new therapies, against say cancer, never randomize people to placebo, which would be unethical because they would die. Instead, they are given the "standard" treatment (like chemotherapy), while the experimental group is given something new.

Before any clinical trial is conducted, doctors and scientists first have to demonstrate that there is sufficient biomedical evidence to justify the clinical trial. But a new study suggests that more than half of clinical trials aren't scientifically justified.

Most Clinical Trials Aren't Scientifically Justified

What constitutes a clinical trial that is scientifically justified? Surprisingly, there are neither stringent guidelines nor widely accepted criteria. This is important because a clinical trial that is not scientifically justifiable is not only a waste of money but also ethically unjustifiable.

So a team of Canadian researchers proposed three criteria for scientific justification of clinical trials in the Journal of Clinical Epidemiology:

1. Is there a specific hypothesis responding to a clear question?
2. Is there uncertainty around the central question?
3. Has this uncertainty been established through a systematic review of the literature?

The first criterion is straightforward: The clinical trial should be hypothesis-driven and address a specific medical need. Drug X should improve the lives of patients who suffer from Disease Y. Giving people large doses of vitamins and seeing if anything good happens to them would be a violation of this criterion.

The second criterion addresses redundancy. For a clinical trial to be justified, there should be unanswered questions about the efficacy of the medical intervention. It is pointless, and therefore unjustifiable, to perform a clinical trial if we already know the answer. For example, large doses of vitamin C don't prevent cancer, so we should not perform any more clinical trials which have this as a primary investigative aim.

The third question concerns research. Have the scientists proposing the clinical trial done their homework? Have they sufficiently studied the literature to fully address criterion #2?

The authors then applied these criteria to 208 clinical trials published in the Journal of the American Medical Association and the New England Journal of Medicine in 2015. They found that 76% of clinical trials satisfied criterion #1, 99% satisfied criterion #2, and 54% satisfied criterion #3. Overall, only 44% of clinical trials met all three criteria.

Limitations and Implications

There are at least two key limitations to the study. The first is that scientists aren't always the best writers, so it is possible that most clinical trials are justified, even if the scientists didn't communicate that fact well. The second is that a new treatment that shows therapeutic potential may not have existed long enough for someone to do a systematic review or meta-analysis. Indeed, it arguably would have been unethical to block Ebola vaccine clinical trials on the grounds that there wasn't yet enough animal data.

Despite these limitations, the authors' criteria could help solve a major problem in biomedical science: The existence of research that cannot be replicated. A bit more thoughtfulness prior to initiating clinical trials could help address this thorny issue, ameliorate ethical concerns, and save a lot of money.

Source: Julie De Meulemeester, Mark Fedyk, Lucas Jurkovic, Michael Reaume, Dar Dowlatshahi, Grant Stotts, Michel Shamy. "Many RCTs May Not Be Justified: A Cross-Sectional Analysis of the Ethics and Science of Randomized Clinical Trials." J Clin Epidemiol (Article in press). Published online: Dec 2017. DOI: 10.1016/j.jclinepi.2017.12.027