I'm Ivermectin Hesitant

Dr. Dan Stein recently presented a dataset representing an ongoing meta-analysis of patients' responses with COVID-19 to ivermectin. You can find the dataset here and his argument here.

Before jumping in, consider the image below. Are there two faces or one vase? Does your perception jump between the two choices? This is a classic figure-ground illusion that I mention as a metaphor for signal and noise. 

Is the vase or the silhouettes the signal? The same can be true for scientific studies. Dr. Stein sees a signal in the dataset on ivermectin; I see noise. Let me explain.

This is a forest plot taken from the dataset. Forest plots are “displays of estimated results from a number of scientific studies addressing the same question.” I have labeled several key features. The line to the left of the number 1 represents equipoise, neither a positive or negative effect of ivermectin. Each of the horizontal lines represents a study with the position of the green square showing the net impact (to the left improvement, to the right no improvement or perhaps detrimental effects). The size of the square reflects the number of participants. The length of the line reflects the confidence interval – longer means that there is less confidence or precision in the data. The diamond above the number 2 is the weighted effect derived from all the studies (squares) above it. In this case, it shows an 81% improvement for people who took ivermectin.

The weighting can be based on the size of the study or its quality. I’m not sure which of these parameters the keepers of the data chose. The general rule of thumb in looking at these studies is that when the horizontal line touches or crosses that vertical equipoise, you cannot draw conclusions with certainty. The asterisk marks the five studies that do not cross the line, as well as the recent study in JAMA that concluded that ivermectin should not be used to treat COVID-19. These are the studies we should more closely consider.

Espitia-Hernandez – This is a small study of 28 patients, given Ivermectin, Vitamin D, and Azithromycin. All were adults over 18, with mild to moderate symptoms, and were PCR- positive (PCR+). When you compare the treated and control groups, you can see they are dissimilar. The treated group has more radiographic evidence of pneumonia, but the control group has lower oxygen levels (As a general rule, I treat patients, not x-rays).

The control group is a bit more febrile, with a faster heart rate and greater body weight than the treatment group. One patient in the control group was hospitalized, none in the treatment group. All members of the treatment group were PCR-negative (PCR-) at ten days, but the controls were not. Was the "success"  In the treatment arm was from ivermectin, Azithromycin (that treats bacterial pneumonia), Vitamin D, or some combination of the three?.  The authors conclude,

Although highly preliminary and probably not sufficiently powered to be conclusive, this proof of concept results supported an effort to evaluate the effect of the proposed combination on the evolution and prognosis of COVID-19 more thoroughly. 

I see mostly noise, little signal, and plenty of uncertainty. But we can agree to disagree.

Bukhari – The study consisted of 86 PCR+ participants, age 15 or older, with mild to moderate symptoms; 50 were assigned to standard care, 50 to standard care, and ivermectin. Ten percent of the control arm and almost 20% of the treatment arm dropped out before the study was completed. The treatment arm showed earlier viral clearance on Day 2, based upon a negative PCR. By day 7, the groups were equal, and by Day 14, more people in the control arm were PCR- than those in the treatment arm. The authors conclude:

In the intervention arm, early viral clearance was observed in patients without experiencing any side effects. These are of importance because high viral load and prolonged viremia can potentially trigger the immune dysregulation phase leading to more severe disease, and the requirement of treatment escalation. 

Being PCR+ does not reflect viral load; you need to know the number of replication cycles necessary to reach that threshold (more cycles, lower viral amounts). All of these patients were hospitalized, and there was no difference in outcome reported other than the PCR conversion. What does it mean when more controls than those treated were PCR- at day 14? For me, yet more noise than a signal.

Elalfy – Non-randomized study of adults with mild to moderate symptoms, PCR+, treated at home, with supportive care or supportive care and a combination of nitazoxanide (a medication used to treat diarrhea of parasitic origin), ribavirin (a BROAD-SPECTRUM antiviral), ivermectin, and zinc. There were distinct differences between the control and treatment groups. The control group had statistically more moderately symptomatic patients with lower baseline oxygenation levels and were more likely to be short of breath than the treatment group before the study began. In the treatment arm, 25% were asymptomatic compared to 4% in the control group before being treated! Finally, 10% of the treatment group and 16% of the control group did not complete the study, making them even less comparable.

Can we agree that there are so many important differences between the treatment and controls that we are comparing apples to oranges? 

The researchers found that PCR conversion to negative was significantly quicker in the treatment arm, 58% versus 13% at day 7. All the patients were hospitalized, and there were no deaths. They conclude,

The results of this study confirm that combined use of nitazoxanide, ribavirin, and ivermectin plus zinc supplement effectively clear SARS-COV2 from the nasopharynx in a shorter time than the symptomatic therapy… 

Can we say the clearance was due to the ivermectin? No. If all the participants remained hospitalized and no one became more clinically ill, does it matter if the PCR converts to negative? I go with clinically interesting, but still more noise than signal. 

Lopez-Medina – This is the controversial JAMA study that concluded that Ivermectin “did not significantly improve the time to resolution of symptoms.” In his analysis, Dr. Stein demonstrated a bias in how the study was performed and analyzed - much as I am attempting to demonstrate with these additional studies. If the data is flawed, then why include what is positive in the Forest plot and dismiss the conclusion? You can't have it both ways; the data cannot be flawed except for the values you think are important. If the study is flawed, let's leave it out.

Chahla – 172 adults over age 18, all PCR+ outpatients with mild symptoms, given supportive care. The treatment group additionally received ivermectin. There was a bit more hypertension, diabetes, and obesity in the treatment group, but not statistically different than the controls. By days 5 to 9, both groups had improved, but the cough and sore throat symptoms were better in the ivermectin group.

Overall, ivermectin resolved symptoms more quickly than standard care alone. More patients were discharged from care by day 28 in the treatment group, but this difference disappeared when stratified by age and gender. There were no deaths in either group. The authors conclude:

Treatment with ivermectin in the population of outpatients with mild stage COVID-19 disease managed to significantly reduce the number of symptoms on days 5 and 9. Subsequent medical examination did not show significant differences.

Fair enough. There’s some signal there. In this study, ivermectin did reduce symptoms. By analogy, it is similar to a "Tamiflu for COVID-19."

Mourya – 100 hospitalized patients age 20 to 60 with mild symptoms and no co-morbidities. Both groups were treated with hydroxychloroquine and azithromycin as the standard of care. The treatment group additionally received ivermectin. 70% of the treatment group were asymptomatic compared to 8% in the controls. In fact, the treatment group was less symptomatic along every parameter; this was a comparison of apples and steaks. Conversion to PCR- was significantly faster in the treatment arm. They conclude,

the treatment with HCQ, azithromycin, and ivermectin had a better success rate compared to HCQ and azithromycin. Based on the results, ivermectin could be the potential therapeutic agents for the COVID-19 disease.

The difference in asymptomatic patients in the groups before the study began was so different that you can draw no meaningful conclusions about treatment. For me, this study is all noise. 

I do not have the time, or inclination, to review the rest of these studies. By my count, nearly a third of the cases I just mentioned should not have been included, but that is my opinion. When you eliminate the studies whose confidence intervals cross the equipoise, you are left with the Chahla study – ivermectin is a COVID-19 version of Tamiflu. That might be useful in the future when COVID-19 doesn’t result in hospitalization.

Speaking as a clinician, I would rather invest my time in vaccination than arguing about this medication. Put it to a randomized controlled study, just like we did for the vaccines. Then we can separate the signal from the noise.