Medical Research’s Cross of ‘Gold’ Imperils Covid Treatments

Randomized controlled studies have advantages, but there are other valuable sources of data

By Allysia Finley. Excerpt:

"“Randomized trials for some purposes is the gold standard, but only for some purposes,” Harvard’s Donald Berwick, a former health adviser to President Barack Obama, said in an interview with GNS Health Care CEO Colin Hill in 2013. “Context does matter. We’re learning in a very messy world, and the context that neatens up that world may make it hard to know how to manage in the real world.”

One obvious problem is that giving patients a placebo when therapies have shown benefits may be unethical. Take the U.K.’s Recovery Trial, which assigns patients either to standard care or different drugs at different stages of the disease. Such adaptive trials allow researchers to enroll more patients and report results faster, and usually mean that fewer patients are given a placebo. The U.K. team reported in June that glucocorticoid dexamethasone reduced mortality in severely ill Covid patients by a third—the most significant finding to emerge from any randomized controlled trial. Some researchers worried that glucocorticoids could dampen the immune response early in the disease, but the treatment was already recommended for severe respiratory distress or infections based on prior trials. Withholding the drug from severely ill patients in a control group might have caused hundreds of unnecessary deaths.  

Randomized trials must usually enroll thousands of subjects to demonstrate a “statistically significant” finding—one that is less than 1% likely to be due to chance. Yet onerous and large trials aren’t always feasible during a pandemic, when information is needed pronto. The U.K. trial, under way since March, still hasn’t reported results on treatments like the monoclonal antibody tocilizumab or convalescent plasma.

As Thomas Frieden, who directed the Centers for Disease Control and Prevention under Mr. Obama, wrote in a 2017 New England Journal of Medicine article: “Elevating RCTs at the expense of other potentially highly valuable sources of data is counterproductive.” Such limitations affect their use for “urgent health issues, such as infectious disease outbreaks.” He added: “No study design is flawless, and conflicting findings can emerge from all types of studies.”

Two randomized trials of Gilead’s antiviral drug remdesivir show how such studies can produce inconclusive results. A randomized trial in China, published in the Lancet in May, enrolled 237 patients. The study found no significant clinical benefit over a placebo, but most of the patients were severely ill when treated. Patients who had symptoms for 10 or fewer days, however, were 25% less likely to die. Similarly, a randomized National Institutes of Health trial with 1,063 patients found the drug reduced average recovery time by four days and the risk of death by 30%, but the survival benefit was statistically insignificant.

Some experts have dismissed the antimalarial hydroxychloroquine, or HCQ, even though more than a dozen observational studies have found it beneficial. A retrospective observational study of Covid-infected nursing-home residents in France, for instance, found those treated with HCQ and azithromycin were 40% less likely to die.

But a few randomized controlled trials found no benefit. A Spanish randomized trial of HCQ for prophylaxis found it didn’t reduce risk of illness among a large group of people exposed in nursing homes, households and health-care settings. Yet two-thirds of the subjects “reported routine use of masks at the time of exposure,” so they were probably less likely to be infected. Nursing-home residents, who may be less likely to wear masks, were 50% less likely to become sick if they took HCQ. But this finding was statistically insignificant, because the trial included only 293 residents.

The U.K. Recovery trial concluded that HCQ was ineffective, but doctors have pointed out flaws. Belgian doctors last month published an observational study that found 32% lower mortality among patients treated with HCQ. They noted that in the U.K. trial “10% of the patients included in the HCQ arm had negative SARS-CoV-2 test and mortality was high in both groups (25%), indicating advanced disease.” Doctors in Italy published an Aug. 25 study finding a 30% lower mortality in hospitalized patients receiving HCQ. They note that the dose in the U.K. trial “was almost the double of that administered in our real life conditions.”

Another problem with Covid-19 randomized trials: Patients at different stages of an illness are often assigned the same dosage. Trials don’t reveal differences in how patients respond to a drug at different dosages or illness severity.

Observational studies can do so. Consider a large study by the Mayo Clinic, which found no overall benefit among patients who received a higher-antibody convalescent plasma versus a lower one. Yet the researchers reported a 37% reduction in mortality among patients under 80 who weren’t on a ventilator and received a high-antibody plasma within three days of hospitalizations."