It takes so much money and time to obtain the necessary permission to do anything (like build housing) that only the big things (like stadiums) are worth attempting

See Sports Stadiums Are Monuments to the Poverty of Our Ambitions by Binyamin Appelbaum of The New York Times. 

It might also be due to the Public Choice idea of "concentrated benefits and dispersed costs." You can get concentrated benefits if you are a team owner so you have an incentive to lobby for public funding. But the costs are dispersed across all the citizens who each only pay a slight amount so it is not worth it to any individual to fight it.

Excerpts:

"Cities build stadiums in part because it’s so hard to build almost anything else. Municipal leaders fixate on big-ticket projects because it takes so much money and time to obtain the necessary permission to do anything that only the big things are worth attempting."

"Washington has 164 different kinds of zoning districts, and even so, many building projects are treated as exceptions. Anyone who wishes to build must run the gantlet."

"Team owners make enough money from stadiums to justify the effort. It’s the smaller projects with slimmer margins that die on the drawing board. In Northwest D.C., there’s a parking lot developers have tried and failed to build on for 25 years."

"Because the system makes it hard to build anything other than luxury projects, cities are increasingly for rich people."

"People spend money at stadiums, but those people overwhelmingly are local residents who otherwise would spend that money in the same community."

"Academic studies have repeatedly concluded that public spending on stadiums is a bad investment. Indeed, one of the leading authorities on the subject has memorably described that conclusion as one of the rare subjects on which economists have approached unanimity."