What the Data Really Says About Women Leaders and the Pandemic

Two new research papers try to shed some light on a popular theory, but the evidence is still very weak—and could point to confirmation bias.

By Hilda Bastian of Wired. Excerpts:

"But jumping from so few examples to the conclusion that the gender of political leaders has been decisive during this pandemic? That just looked to me like confirmation bias.

The theory, in its standard form, skated over certain inconvenient facts; for instance, how early it was in the pandemic, and how badly some “women-led” countries were doing. One particularly prominent story—with more than 8 million views as I’m writing—came out on April 13 in Forbes. The author, a professional gender-balance consultant, made her case with a few cherry-picked countries. Not mentioned? The fact that Belgium, led by Sophie Wilmès, was notching up the world’s highest rate of Covid-19 deaths per million population for any country (other than a microstate in northern Italy). Another story in this genre began by singling out Silveria Jacobs, the prime minister of Sint Maarten in the Caribbean, for her exemplary handling of the pandemic’s risks. According to Worldometer, Sint Maarten currently has the twelfth-worst rate of Covid-19 mortality per million among all listed countries and territories. Given that women make up about 10 percent of national leaders, the presence of Wilmès and Jacobs on the bottom-20 list for this key metric doesn’t support the thesis that women leaders are doing any better (or any worse) than men.

Women leaders are still so unusual that they stand out and draw a lot of scrutiny. I needed more than a few high-profile cases of their success—Ardern, Merkel, Taiwan’s Tsai Ing-Wen, etc.—to have an opinion about the women pandemic better narrative. So a few weeks ago I did some very crude calculations based on data sources that are themselves pretty crude. For example, I used Wikipedia’s list of 22 current elected or appointed female heads of state and government, without differentiating whether each woman was a governing leader, such as Norway’s prime minister Erna Solberg, or serving in more of a titular role, such as Slovakia’s president Zuzana Čaputová. (The media narratives mostly avoid this distinction, too.) The “women-led” countries were not more likely to have below-average mortality rates per million population. That doesn’t answer the question about leadership performance, of course, but it left me skeptical.

Now we have more formal data. Two sets of academics tried to analyze differences in Covid-19 outcomes among countries with men and women leaders, and they posted their results as preprints in June. Each concluded that countries led by women have done better. But not only were both studies vulnerable to bias; neither found a statistically significant overall difference based on gender. Their approaches could not overcome the fundamental problems caused by the small sample size of women leaders.

The first study, posted on June 3 by Supriya Garikipati from the University of Liverpool and a colleague, did try to distinguish between women leaders in governing and less powerful roles. While they didn’t find a significant difference in Covid-19 cases and deaths based on leaders’ gender, they did find an effect after using modeling to match pairs of male- and female-led countries with similarly sized elderly populations, health care expenditures, and openness to tourism, among other factors. They don’t report enough data to assess the result, though—not even which 19 countries they judged to be “female-led.” This is the kind of study that badly needs to have a protocol set down, and available for scrutiny, before running any analyses at all. In the absence of that step, a reader would never know whether the choice of factors in the final model was changed midstream or whether unfavorable analyses were done but not reported. What’s more, with so many factors in their analyses of this small group of countries, the risk of coincidental associations is perilously high; and, at the same time, other issues that could be important weren’t included in the model at all, such as whether a country is an island nation.

The second study, posted June 12 by Soumik Purkayastha and colleagues at the University of Michigan, did what I had done—use Wikipedia’s list of female heads of state and government without differentiating between types of leaders. They excluded countries with fewer than 100 people with confirmed Covid-19, and ended up with data for 18 women-led countries. That approach adds bias in favor of women leaders, because from what I can see, the method disproportionately excludes “male-led” countries that managed to keep the coronavirus at bay.

It’s easy to find examples of how tricky it can be to make sweeping claims based on these small numbers. Four out of the five major Nordic countries are led by women, who in turn make up a sizable chunk of the women in the recent studies and media narratives described above. It’s often pointed out that the Nordic countries led by women—Denmark, Norway, Finland, and Iceland—have contained the pandemic much better than the one led by a man, Sweden. But what difference would it have made if Sweden’s prime minister had been a woman? Swedish law forbids the country’s political leaders from overruling a recommendation from its public health agency, and that’s where the pandemic control decisions were made."