Diagnostic test kits might have shown that fewer people actually had the disease than the study claimed–what Wachter calls an “overcall,” something that might be more likely in a place like San Francisco, where the overall prevalence of infection was low. Just 4 percent of the people who came to UCSF with Covid-like symptoms in March actually had Covid-19. But in this case, he says, even if that was the case, it wouldn’t have biased the final results. “There’s no good reason to believe that this wouldn’t lead to more cases of Covid being called equally in both groups,” Wachter says, “so it should have no ultimate effect on the study.”
It’s possible, too, that the overall number of people the study looked at wasn’t big enough to pick up a smaller but still positive effect. “The study was designed to be large enough to detect a halving in the risk of infection–and they did not find one. However, their trial could not rule out a more modest difference, which would still be very valuable,” emails Martin Landray, a researcher in the University of Oxford’s Nuffield Department of Population Health and one of the people running the UK’s massive Recovery trial of Covid-19 drugs, including hydroxychloroquine. “So, a small, statistically underpowered randomized trial of an important question. It excludes a large effect (benefit or harm) of HCQ but leaves open the possibility that there may be more modest effects (or none at all).
On a positive note, no matter whether they received the placebo or the drug, only around 10 percent of the study participants exposed to the disease actually got infected–in keeping with overall estimates from the Centers for Disease Control and Prevention. This fact supports the idea that so-called superspreading events have been critical to the disease’s transmission; that whether thanks to some physiological or immunological variation or environmental circumstances, only a small number of people spread the disease to a vastly larger number of people, who account for most new infections. (On the other hand, this could mean that the people in the trial had a lower risk of infection, says Muge Cevik, an infectious disease researcher at the University of St. Andrews. “I think it’s still an open question whether post-exposure prophylaxis would be beneficial if given, for example, to a very high-risk group,” she says.)
Despite the fears of dangerous side effects from earlier studies, Boulware’s trial revealed only mild ones, if any–nausea and diarrhea mostly. That happened to about 40 percent of people taking the drug, but 17 percent of people on the placebo, too.
Does this study spell the end for the idea of prophylaxis? After all, it did not show that the drug prevented infection among people exposed to Covid-19 patients. Boulware allows that the right dose might still prevent the disease if administered before exposure–pre-exposure prophylaxis, for people who haven’t been around anyone who’s infected with Covid-19, but who might be later. And since this trial only looked at prevention, not treatment, it’s still possible to imagine that it would work just well enough as a treatment to make any side effects worth it. Boulware is still working on a paper talking about whether the drug works as a treatment for those who are already ill. But all that is hypothetical if it doesn’t work at all.
So hydroxychloroquine might not be entirely off the table. It’s the nature of science not to be utterly airtight. “No one study is the final word, especially when other trials are ongoing–and it’s not like the percentages with the outcome were exactly the same in the two groups here. The correct interpretation is as the authors wrote, which is that this study does not demonstrate benefit in prophylaxis,” says Walid Gellad, director of the Center for Pharmaceutical Policy and Prescribing at the University of Pittsburgh. “But it is incorrect to say that this study proves it doesn’t work.”
