A Closer Look at Climate Model Performance

From Paul C. "Chip" Knappenberger and Patrick J. Michaels of Cato. 

"The Current Wisdom is a series of monthly articles in which Patrick J. Michaels and Paul C. “Chip” Knappenberger, from Cato’s Center for the Study of Science, review interesting items on global warming in the scientific literature or of a more technical nature. These items may not have received the media attention that they deserved or have been misinterpreted in the popular press.
 

—

Posted Wednesday in the Washington Post’s new online “Energy and Environment” section is a piece titled “No, Climate Models Aren’t Exaggerating Global Warming.” That’s a pretty “out there” headline considering all the evidence to the contrary.

We summed up much of the contrary evidence in a presentation at the annual meeting of the American Geophysical Union last December.  The take-home message—that climate models were on the verge of failure (basically the opposite of the Post headline)—is self-evident in Figure 1, adapted from our presentation.



Figure 1. Comparison of observed trends (colored circles according to legend) with the climate model trends (black circles) for periods from 10 to 64 years in length. All trends end with data from the year 2014 (adapted from Michaels and Knappenberger, 2014).

The figure shows (with colored circles) the value of the trend in observed global average surface temperatures in lengths ranging from 10 to 64 years and in all cases ending in 2014 (the so-called “warmest year on record”). Also included in the figure (black circles) is the average trend in surface temperatures produced by a collection of climate models for the same intervals. For example, for the period 1951–2014 (the leftmost points in the chart, representing a trend length of 64 years) the trend in the observations is 0.11°C per decade and the average model projected trend is 0.15°C per decade. During the most recent 10-year period (2005–2014, rightmost points in the chart), the observed trend is 0.01°C per decade while the model trend is 0.21°C per decade.

Clearly, over the period during which human-caused greenhouse gases have risen the fastest (basically any period ending in 2014), climate models consistently predict that the earth’s surface temperature should have warmed much faster than it did.

Given our results (and plenty like them), we were left scratching our heads over the headline of the Post article. The article was reporting on the results of a paper that was published last week in the British journal Nature by researchers Jochem Marotzke and Piers Forster, and pretty much accepted uncritically what Marotzke and Forster concluded.

The “accepted uncritically” is critical to the article’s credibility.

Figure 2 shows the results that Marotzke and Forster got when comparing observed trends to model-predicted trends of lengths of 15 years for all periods beginning from 1900 (i.e., 1900–1914) to 1998 (1998–2012). Marotzke and Forster report that overall, the model trends only depart “randomly” from the observed trends—in other words, the model results aren’t biased.

But this claim doesn’t appear to hold water.

During the first half of the record, when greenhouse gas emissions were relatively small and had little effect on the climate, the differences between the modeled and observed temperatures seem pretty well distributed between positive and negative—a sign that natural variability was the driving force between the differences. However, starting in about 1960, the model trends show few negative departures from the observations (i.e., they rarely predict less warming than was observed). This was partially due to the model mishandling of two large volcanic eruptions (Mt. Agung in 1963 and Mt. Pinatubo in 1992), but also it is quite possibly a result of the models producing too much warming as a result of increasing greenhouse gas emissions. It seems that the models work better, over the short term (say 15 years), when they are not being forced by a changing composition of the atmosphere.



Figure 2. Comparison of observed trends (black) with the climate model average trend (red) for periods of 15 years in length during the period 1900–2012 (adapted from Marotzke and Forster, 2015).

But the models appear to do worse over long periods.

Figure 3 is also from the Marotzke and Forster paper. It shows the same thing as Figure 2, but this time for 62-year-long trends. In this case, the models show a clear and persistent inability to capture the observed warming that took place during the first half of the 20^th century (the models predict less warming than was observed over all 62-year periods beginning from 1900 through 1930). Then, after closely matching the observed trend for a while, the models began to overpredict the warming beginning in about 1940 and progressively do worse up through the present. In fact, the worst model performance, in terms to predicting too much warming, occurs during the period 1951–2012 (the last period examined).



Figure 3. Comparison of observed trends (black) with the climate model average trend (red) for periods of 62 years in length during the period 1900–2012 (adapted from Marotzke and Forster, 2015).

This behavior indicates that over longer periods (say, 62 years), the models exhibit systematic errors and do not adequately explain the observed evolution of the earth’s surface temperature since the beginning of the 20^th century.

At least that is how we see it.

But perhaps we are seeing it wrong.

Over at the website ClimateAudit.org, Nic Lewis (of low climate sensitivity fame) has taken a very detailed (and complicated) look at the statistical methodology used by Marotzke and Forster to arrive at their results. He does not speak in glowing terms of what he found:

“I was slightly taken aback by the paper, as I would have expected either one of the authors or a peer reviewer to have spotted the major flaws in its methodology.”

“Some statistical flaws are self evident. Marotzke’s analysis treats the 75 model runs as being independent, but they are not.”

“However, there is an even more fundamental problem with Marotzke’s methodology: its logic is circular.”

Lewis ultimately concluded:

“The paper is methodologically unsound and provides spurious results. No useful, valid inferences can be drawn from it. I believe that the authors should withdraw the paper.”

Not good.

So basically no matter how* you look at the Marotzke and Forster results—taking the results at face value or throwing them out altogether—their conclusion are not well-supported. And certainly, they are no savior for poorly performing climate models.

*No matter how, that is, except if you are looking to try to make it appear that the growing difference between climate model projections and real world-temperature change poses no threat to aggressive measures attempting to mitigate climate change.

References:

Marotzke, J., and P. Forster, 2015. “Forcing, Feedback and Internal Variability in Global Temperature Trends.” Nature, 517, 565–570, doi:10.1038/nature14117.

Michaels, P.J., and P.C. Knappenberger, 2014. “Quantifying the Lack of Consistency Between Climate Model Projections and Observations of the Evolution of the Earth’s Average Surface Temperature since the Mid-20th Century.” American Geophysical Union Fall Meeting, San Francisco, CA, Dec. 15–19, Paper A41A-3008."