Infrastructure Projects: History of Understated Costs and Overstated Benefits

From Timothy Taylor.

"It sometimes seems as if every big infrastructure project underestimates its costs and overpromises its benefits. But is that a few bad apples that get a lot of publicity, or is it the real overall pattern? Bent Flyvbjerg and Dirk W. Bester put together some evidence in “The Cost-Benefit Fallacy: Why Cost-Benefit Analysis Is Broken and How to Fix It” (Journal of Benefit-Cost Analysis, published online October 11, 2021).

They collect “a sample of 2062 public investment projects with data on cost and benefit
overrun. The sample includes eight investment types: Bridges, buildings, bus rapid
transit (BRT), dams, power plants, rail, roads, and tunnels. Geographically, the
sample incorporates investments in 104 countries on six continents, covering both
developed and developing nations, with the majority of data from the United States
and Europe. Historically, the data cover almost a century, from 1927 to 2013.” Not all of these studies have data on both expected and realized benefits and costs. But here’s a table summarizing the results: On average, there are cost overruns in every category, and overstated benefits in every category. In more detailed results, they show that this pattern hasn’t evolved much over time.

Of course, the average doesn’t apply to every project. Indeed, sometimes there are cost overruns but then even bigger benefits than expected. But the average pattern is disheartening. Indeed, “[c]onsidering cost and benefit overrun together, we see that the detected biases work in such a manner that cost overruns are not compensated by benefit overruns, but quite the opposite, on average. We also see that investment types with large
average cost overruns tend to have large average benefit shortfalls.”

The essential problem here, Flyvbjerg and Bester argue, is that those who do these estimates of benefits and costs are overly optimistic.

The root cause of cost overrun, according to behavioral science, is the well-documented fact that planners and managers keep underestimating the importance of schedules, geology, market prices, scope changes, and complexity in investment after investment. From the point of view of behavioral science, the mechanisms of scope changes, complex interfaces, archeology, geology, bad weather, business cycles, and so forth are not unknown to public investment planners, just as it is not unknown to planners that such mechanisms may be mitigated. However, planners often underestimate these mechanisms and overestimate the effectiveness of mitigation measures, due to well-known behavioral phenomena like overconfidence bias, the planning fallacy, and strategic misrepresentation.

Thus, the question is how to get those estimating the benefits and costs of mega-projects to be more realistic. The authors offer some suggestions.

“Reference class forecasting” is the idea of basing your estimates on look at actual costs and benefits that happened with similar projects in other places, representing a range of better and worse outcomes. Another idea is to give the benefit-cost forecasters some “skin in the
game.” “Lawmakers and policymakers should develop institutional setups that reward
forecasters who get their estimates right and punish those who do not.” This can be done in friendly ways, with bonuses, or it can be done in punitive ways, with lawsuits and even crimial punishments when things go badly wrong. There can be a rule in advance that independent audits will be carried out during and after the project–perhaps even by several different auditors. Finally, the decisions about whether to proceed shouldn’t just involve technocrats and spreadsheets, but also need public involvement. For example, if the public is going demand processes that slow down or complicate a project, that needs to be taken into account at the start–even if those demands may seem irrelevant or counterproductive to the forecasters.

The authors note that these kinds of changes are being used in various places and by various governments around the world. If there are plans for a megaproject where you live, you might want to think about whether these processes might lead to more accurate benefit-cost estimates, too."