The fallibility of the CBO

A supposed weakness in the barrier between the spirit and material worlds made Halloween a time when people tried to predict the future. By throwing wet hazelnuts into a fire, melting lead into cold water or even bobbing for apples, people tried to figure out what would happen in the year ahead. These methods may seem quaint, but given the inaccuracy of the Congressional Budget Office’s (CBO) projections, perhaps the agency should drop the Keynesian economic models and invest in a barrel of apples instead. 

{mosads}Whether it scores the recent healthcare bill or the cost of the Capitol Hill Visitor Center, an ambitious three-floor underground facility, the price for taxpayers always ends up larger than originally predicted. 

Take Medicare spending: the CBO’s long-term projections of Medicare spending have steadily increased, even in recent years and over short periods of time. In 2005, the CBO projected that Medicare would cost $1.5 trillion in 2050. Two years later the same CBO projected that this cost would reach $2.8 trillion in 2050. And in 2009, it projected that it would be $3 trillion instead. The program’s projected cost doubled in four years.

To be sure, the law that tells CBO how to score bills is partially responsible for this. Under current law, the agency must score bills as written by Congress, which means that it has to include the unrealistic assumptions and gimmicks adopted by lawmakers. That’s how, in the case of the Affordable Care Act (ACA), the CBO ended up predicting the bill would reduce the deficit by $143 billion over the next 10 years. Many experts were skeptical. 

To its credit, CBO was skeptical too, noting, “these projections assume that the proposals are enacted and remain unchanged throughout the next two decades, which is often not the case for major legislation.” The agency even produced an alternative scenario.

And then of course, there is politics. As budget guru Stan Collender recently emailed me, “CBO and its directors have been heavily criticized literally since the organization was established by the Congressional Budget Act. CBO’s first director — Alice Rivlin — was repeatedly condemned when she said things that Congress didn’t like.”

But the CBO isn’t without responsibility for its failed predictions. Like many economists, its analysts suffer from a misplaced belief in their forecasting prowess. For instance, in the case of the ACA predictions, CBO was only skeptical about lawmakers’ ability to implement the law as written — not about the fallibility of its model.

It should be. CBO relies heavily on Keynesian economic models, like the ones it used during the stimulus debate. Forecasters at the agency predicted the stimulus package would create more than 3 million jobs. And in August 2010, the CBO estimated that the stimulus had indeed created between 1.4 million and 3.6 million extra jobs.

But unemployment stubbornly remained around 10 percent. What was wrong with the CBO’s numbers? 

“When the upper limit of your estimate is almost three times the lower limit, you know it is not a very precise estimate,” George Mason University economist Russ Roberts emphasized in congressional testimony in February.

The truth is, there is no way to know the real number of jobs “created or saved” by the stimulus. For that, the CBO would have had to collect data on output and employment while holding other factors constant. But the CBO didn’t do that, because that’s different from its job of “scoring” the possible results of proposed legislation. As the CBO explained in a November 2009 report: “Isolating the effects would require knowing what path the economy would have taken in the absence of the law. Because that path cannot be observed, the new data add only limited information about [the law’s] impact.” 

In other words, CBO number crunchers gave it their best guess.

No one knows what economic output would have been without the stimulus, and no models can tell us the answer. But this lack of firm knowledge didn’t stop CBO Director Douglas Elmendorf from talking as though he knew for a fact that these stimulus jobs had materialized.

Worst of all, it won’t stop there. The CBO will keep predicting and all of us will keep using the predictions as if they were accurate. Yet, the stimulus and the ACA should serve as yet more evidence that Congress should take budget scores and economic projections with a grain of salt. What looks good in the spirit world of the computer model may be very bad in the material realm of real life because people react to changes in policies in ways unaccounted for in these models.

As everyone is predicting the future, I think I’ll join in: CBO will continue to make inaccurate predictions and Congress will continue to rely upon them, passing expensive bills that will drag down the economy even further. And that should scare you, even at Halloween.

De Rugy is a senior research fellow at the Mercatus Center at George Mason University.