Economic Forecasting Is Really Difficult. Just Ask the IMF
The world economy will be less vibrant this year than had been expected three months ago. That’s according to the International Monetary Fund’s new World Economic Outlook report, published Tuesday, which projects global 2019 gross domestic product will grow 3.3 percent, 0.2 percentage point lower than in forecasts in January. For the world’s two biggest economies, the revisions moved in opposite directions of each other, with the U.S. growth forecast lowered 0.2 percentage point to 2.3 percent while China’s rose 0.1 percentage point to 6.3 percent.
The IMF’s forecasts are some of the most-watched in the world and routinely make for essential reading among policy makers, economists and traders. So how much confidence should we place in these forecasts?
A Bloomberg analysis of more than 3,200 same-year country forecasts published each spring since 1999 found a wide variation in the direction and magnitude of errors. In 6.1 percent of cases, the IMF was within a 0.1 percentage-point margin of error. The rest of the time, its forecasts underestimated GDP growth in 56 percent of cases and overestimated it in 44 percent. The average forecast miss, regardless of direction, was 2.0 percentage points, but obscures a notable difference between the average 1.3 percentage-point error for advanced economies compared with 2.1 percentage points for more volatile and harder-to-model developing economies. Since the financial crisis, however, the IMF’s forecast accuracy seems to have improved, as growth numbers have generally fallen.
Forecast Errors, Country by Country 👆
Advanced economies
Emerging and developing economies
IMF forecasts matter. Its network of economists covers most of the globe, with 193 countries included in this analysis. Its regular economic outlooks influence decisions taken by governments and private organizations worldwide—surveys show that top officials in central banks and finance ministries in a majority of countries use IMF projections as a benchmark or to check their own forecast accuracy. The organization is also an emergency lender for nations in crisis. When the IMF gets it wrong, there can be real costs.
“Economic forecasting—trying to predict future movements of the economy—is by definition an exercise fraught with uncertainty,” said IMF chief spokesman Gerry Rice in a written comment provided to Bloomberg News, adding that it’s a challenge faced by all economists and institutions. “In that sense, subtracting the projections from actual growth outcomes seems overly simplistic, and does not do justice to the complexities and uncertainties related to the exercise.” (The IMF’s internal assessments of forecast accuracy have used similar calculations in the past.)
“Our objective with our forecasts is to provide as accurate a figure as possible reflecting the information available at that time about the economy, taking into account assumptions regarding its likely evolution and policy choices,” said Rice.
When compared with private forecasters, the IMF’s own analysis has found that it’s no more or less accurate, as laid out in an unvarnished 2014 report published by its own Independent Evaluation Office. The report also found no evidence of “substantial positive or negative biases,” yet a more detailed look at the direction of forecast errors reveals how the fund has at times demonstrated a tendency to be too gloomy or too rosy for specific countries over the past 20 years. For instance, the IMF overestimated U.S. growth 80 percent of the time and China’s growth 20 percent of the time.
However, the errors aren’t always as bad as they seem. Whereas U.S. growth was technically overestimated in 16 of 20 years, that includes errors in three recent years of less than 0.1 percentage point. And in China, which had the exact opposite long-term pattern, there’s a far less pronounced underestimation tendency when looking at the last 10 years. As for the IMF’s relatively large forecast misses in the early 2000s, these are partly explained by major backward revisions that China has made to its growth numbers over time; the 2007 growth figure was revised to 14.2 percent, from an initial 11.4 percent.
How 3,284 Predictions Panned Out 👆
- Average of all countries, unweighted
- Selected country
The IMF’s track record of forecasting China is reflected in another curious phenomenon: By and large, Asian countries were underestimated by the IMF more than any other region in the world. A prior study by the IMF of its forecasts examined whether economic interdependencies between countries are properly taken into account and found that “forecasts of the Chinese economy appear to contain information that could have improved other forecasts.”
Regional Misses
- No data
62.7%
60.4%
47.0%
78.4%
65.0%
46.3%
60.1%
46.8%
39.2%
59.6%
59.1%
49.1%
58.5%
61.4%
62.7%
47.0%
E. Europe
60.4%
Other more developed regions
W. Europe
78.4%
65.0%
C. Asia
E. Asia
46.3%
59.1%
46.8%
N. Africa
39.2%
W. Asia
Oceania
Caribbean
59.6%
C. America
49.1%
60.1%
Sub-Saharan Africa
S. Asia
61.4%
58.5%
S.E. Asia
S. America
62.7%
Eastern Europe
60.4%
47.0%
Western Europe
78.4%
Other more developed regions
C. Asia
65.0%
Eastern Asia
60.1%
46.3%
S. Asia
Northern Africa
46.8%
39.2%
Oceania
Caribbean
59.6%
Central America
59.1%
49.1%
Western Asia
Sub-Saharan Africa
61.4%
South-Eastern Asia
58.5%
South America
62.7%
Eastern Europe
47.0%
60.4%
Other more developed regions
78.4%
Western Europe
Central Asia
65.0%
Eastern Asia
60.1%
46.3%
59.1%
Southern Asia
Northern Africa
Western Asia
59.6%
46.8%
39.2%
Central America
Oceania
Caribbean
49.1%
Sub-Saharan Africa
61.4%
58.5%
South-Eastern Asia
South America
62.7%
Eastern Europe
60.4%
78.4%
Western Europe
Central Asia
47.0%
Other more developed regions
65.0%
Eastern Asia
60.1%
59.1%
46.3%
Southern Asia
Western Asia
Northern Africa
59.6%
Central America
39.2%
Caribbean
46.8%
Oceania
49.1%
Sub-Saharan Africa
61.4%
South-Eastern Asia
58.5%
South America
| Region | % of Forecasts that were Underestimates% of Underestimates |
|---|---|
| Central Asia | 78.4 |
| Eastern Asia | 65.0 |
| Eastern Europe | 62.7 |
| South-Eastern Asia | 61.4 |
| Western Europe | 60.4 |
| Southern Asia | 60.1 |
| Central America | 59.6 |
| Western Asia | 59.1 |
| South America | 58.5 |
| Sub-Saharan Africa | 49.1 |
| Other more developed regions | 47.0 |
| Oceania | 46.8 |
| Northern Africa | 46.3 |
| Caribbean | 39.2 |
| World | 55.7 |
Beyond the issue of interdependent economies, the IMF’s report stressed that forecasters, including itself, “tend to overpredict GDP growth significantly during regional or global recessions, as well as during crises in individual countries” and that “they tended to be optimistic in high-profile cases characterized by exceptional access to IMF resources.”
There are perhaps no better examples of recent “high-profile” cases than the multiple European debt crises that struck the continent at the start of this decade. Banking and sovereign debt panics hit Greece, Ireland, Portugal and Cyprus to varying degrees, threatening the integrity of the euro area and requiring emergency intervention from multinational authorities.
During this period, the IMF wasn’t merely forecasting what would happen to these countries but also setting the terms. It provided billions in bailout loans in exchange for implementation of strict austerity measures and other policies, often bitterly opposed by the countries’ citizens and politicians. Importantly, the IMF was just one of three institutions structuring the rescue packages: The European Union and European Central Bank made up the other two partners of the so-called “troika.”
These complicated relationships with the borrower countries may well have made the forecasting task all the more difficult. For instance, the IMF’s initial forecast for Greece proved far too rosy in retrospect, and even subsequent-year forecasts that reflected the IMF’s lowered expectations proved too optimistic. The fund’s record on Portugal mirrored that toward Greece in the first two years of its crisis, but came much closer to the mark from 2014 onward. Meanwhile, Ireland and Cyprus outperformed the fund’s predictions, recovering relatively quickly; Ireland even repaid the IMF ahead of schedule.
The Changing Outlook for Bailout Countries
- Actual GDP growth
- Multiyear forecasts
It won’t be possible to test the accuracy of Tuesday’s forecasts for a while, but chances are the results will bear out the reality that forecasting developed economies is simply a more straightforward process. In general, growth is lower and more stable, and data coverage is more comprehensive. But even the most meticulous forecast can be thrown off by the unexpected.
“Unforeseen developments, such as a conflict, natural disasters or other economic shocks, can have major implications for the overall forecast,” said Rice. “This is not to say that forecasting can’t be improved. We also subject our forecasting methods to regular and stringent evaluations, both internally and externally.”