Catalog

	Table of Contents
p. 36 of 36 BACK | NEXT Foreword 1. Developing Asia and the world II. Economic trends and prospects in developing Asia III. Developing Asia's imprint on global commodity markets Appendix Statistical notes and tables ADO forecasting performance for GDP growth and inflation

Asian Development Outlook 2006 Update

---

ADO forecasting performance for GDP growth and inflation

Asian Development Outlook (ADO) has tended to underpredict growth and overpredict inflation, according to analysis of its track record in forecasting these indicators. Its forecast errors are most pronounced at times of large swings in economic growth. Forecasting performance has improved since the end of the Asian crisis, though directions of bias remain the same. Also, ADO's growth and inflation forecasts have generally been more accurate than Consensus Forecasts for the group of countries covered in the Update.

Introduction

The Asian Development Bank (ADB) has been publishing its assessment of the economic prospects for its developing member countries (DMCs) since 1989 in ADO. This is released in April of each year, and its Update (since 2000) has appeared in September the same year. The ADO country economic analysis and projections provide important context for ADB’s policy dialogue with its DMCs and other developing partners, and guides ADB management in its public commentary about economic developments and other important policy issues.

It is important therefore to review ADO's forecasting performance and to consider ways in which their accuracy and usefulness can be improved. Readers need to be mindful that forecasts are prone to errors. An investigation of past errors may provide useful information that will permit future improvements.

Measuring the quality of forecasts

This exploratory investigation will examine errors in forecasts of real GDP growth and inflation, as these generally receive more public interest and attention than other economic variables. Current-year and next-year forecasts of nine DMCs of ADB, from 1990 to 2005, are considered. (The nine DMCs are those covered in this Update: Bangladesh, India, Indonesia, Malaysia, Pakistan, People’s Republic of China [PRC], Philippines, Thailand, and Viet Nam.) The ADO forecasts are then compared with the April release of Consensus Forecasts from Consensus Economics (which polls forecasts on a monthly basis) for the post-crisis period from 1999 to 2005.

One measure of the accuracy of forecasts is the size of the forecast error or the difference between actual values and the forecast for a given year. For example, the current-year forecast error in GDP growth in 2005 is the difference between actual growth in 2005 and the forecast made in ADO 2005. As the ADO 2005 forecast published in April is formulated in March, the forecast has about a 9-month lead to end-2005. The next-year forecast error is the difference between actual growth in 2005 and the forecast made in ADO 2004. Positive forecast errors indicate underprediction and negative forecast errors indicate overprediction.

Although the mean forecast error is widely used as a simple measure of accuracy, it can be misleading if positive and negative values cancel each other out. Thus, a small mean forecast error can result either from the fact that all errors are small or if all the errors are large, because underestimates and overestimates cancel each other out. To overcome these limitations, one can look at alternative measures that disregard the arithmetic signs—the mean absolute error (MAE) and the root-mean-square error (RMSE). These show the magnitude of the errors without regard for the sign, and the latter gives greater weight to larger errors.

ADO's forecasting record

Table 1 reports basic statistics for the forecast errors for both current-year and next-year forecasts of GDP growth and inflation for each of the nine DMCs. The mean, median, standard deviation, coefficient of the 1st-order serial correlation of the forecast errors, the fraction of positive errors, the MAE, and the RMSE are shown.

GDP growth

Current-year forecasts. For GDP growth, the mean of the current-year forecast error (i.e., the bias averaged across time and across countries) is close to zero. The PRC posted the largest forecast error—close to 2 percentage points. This indicates that the PRC GDP growth forecasts are predominantly underestimated. Mean forecast errors for other countries are also positive (i.e., growth is underpredicted) but are less than 1 percentage point (Bangladesh, India, Malaysia, and Viet Nam). On the other hand, forecasts for the Philippines, Thailand, Pakistan, and Indonesia have negative errors (i.e., growth is overpredicted). With the exception of the Philippines, GDP growth forecasts for the three countries are higher than actual data by less than half a percentage point. On average, ADO forecasts of Philippine GDP growth are overestimated by 0.6 percentage points. This is largely due to the large negative errors during periods when the country faced economic turmoil (i.e., 1991 economic recession, and the 1997–98 Asian financial crisis). For countries not affected by outliers, the biases are low. This is true for Bangladesh and India where mean forecast errors are close to zero.

Due to the sensitivity of the mean forecast error to extreme values, it is useful to consider the median forecast error and the fraction of positive errors. A case in point is Indonesia, where the mean forecast error is negative while the median forecast error and more than 80% of the forecast errors are positive. This is because its mean forecast error is significantly influenced by a few large negative errors (which occurred at the time of the Asian crisis). The same is true for Thailand. The standard deviations of the forecast errors for these two countries are high.

Except for the Philippines, a majority of the forecasts for each country’s GDP growth are underpredicted. A particularly high fraction of positive errors are observed for the PRC, Indonesia, Malaysia, and Viet Nam. Consistent with this, the median forecast errors are large and positive.

Next-year forecasts. Next-year mean forecast errors are negative except for the PRC and Viet Nam. The observation that next-year’s overall mean forecast error is smaller and hence more accurate than the current-year is misleading, as large positive and negative forecast errors are canceling out. Disregarding arithmetic signs, the MAE and the RMSE are larger for next-year forecasts. The PRC, Philippines, Pakistan, and Thailand all report forecast errors exceeding 1 percentage point. The negative bias of forecast errors for Bangladesh, Pakistan, Philippines, and Thailand and the positive bias for PRC are notable.

The proportion of positive next-year forecast errors is very low for Bangladesh (0.25), Pakistan (0.25), and the Philippines (0.38). But just as for current-year forecasts, next-year forecasts for the PRC, Indonesia, Malaysia, Thailand, and Viet Nam have a large proportion of positive forecast errors, reflecting a strong tendency to underpredict GDP growth.

As next-year forecasts are more susceptible to unexpected economic developments, errors are more volatile when compared with current-year forecasts. The standard deviations of next-year forecast errors are greater than those of the current year. In particular, the increase of the spread in errors between the current-year and the next-year forecasts for many of the countries appears quite large, and was, for Indonesia for example, almost double.

Inflation

Current-year forecasts. A handful of countries have experienced high inflation rates: PRC, India, Pakistan, Philippines, and Viet Nam during the early part of the 1990s and Indonesia and Thailand during the latter part. Indonesia and Viet Nam recorded the two highest inflation rates. Consequently, mean forecast errors for these two countries were significantly higher than for other countries. For Indonesia, current-year forecasts for inflation were underpredicted by 2.7 percentage points. This was largely driven by the failure to anticipate the positive effect on inflation in 1998 of the Asian financial crisis. For Viet Nam, current-year inflation estimates were, on average, above outcomes by 4.4 percentage points. This is largely explained by the forecast errors made in 1990 and 1991, when inflation was high.

A negative overall mean forecast error suggests a general tendency for ADO to overpredict inflation rates. But again this measure may be affected by the presence of outliers. With the exception of India and Indonesia, the majority of the countries’ forecast errors are negative. Taken as a whole, almost 60% of the countries’ forecast errors are negative, suggesting a general tendency of current-year forecasts of the ADO to overpredict inflation.

Next-year forecasts. A somewhat similar picture emerges for the next-year forecast errors. The mean forecast errors and the proportion of positive errors suggest a general tendency for next-year forecasts to overpredict inflation rates. Taken as a whole, the proportion of positive errors is only a little bit below 50%. The proportions of positive errors are low for the PRC (0.31), Malaysia (0.19), and Viet Nam (0.4). It is only India and Indonesia that have high proportions of positive errors (0.56 and 0.75, respectively) while remaining countries have equal proportions of positive and negative forecast errors.

Although the mean forecast error for the next-year forecasts may appear to be lower than the current-year forecasts, this is again because of self-cancellation of large positive and negative errors. An examination of the MAE and the RMSE statistics confirms this. Next-year forecasts’ MAE is 4.1 percentage points compared with the current-year forecasts’ MAE of 2.1 percentage points. Furthermore, standard deviations of next-year forecast errors are higher than current-year forecast errors.

Serial correlation appears to be a problem for the forecast errors for PRC, India, and Viet Nam. In the case of PRC and Viet Nam, the problem appears in both current- and next-year forecast errors.

Forecast errors and volatility in growth and inflation

Figures 1 and 2 plot annual change in GDP growth and inflation against current- and next-year forecast errors. The figures show the relationship between forecast errors and year-to-year changes in growth and inflation.

From Figures 1 and 2, it can be seen that forecasts tend to miss the mark most when there are sharp changes in outcomes. For example, GDP growth was significantly overpredicted for the Asian financial crisis years: the 1997 current-year forecasts by about 2.2 percentage points, and the 1998 current-year forecasts by about 3.6 percentage points. Next-year forecast errors were even larger.

The figures show the presence of large forecast errors during positive and negative swings for both growth and inflation. But generally, forecast errors are larger at the time of downturns than upturns. The reasons for this are not immediately apparent, but would be consistent with a reticence to add to bad news.

As regards inflation, large overpredictions were made not only during the Asian financial crisis but also during the early 1990s, when many countries were adversely affected by generally slower growth in the world economy and higher petroleum prices following the onset of the Gulf crisis.

Although ADO forecasts generally follow the ups and downs of GDP growth and inflation, there would appear to be a bias in the forecasts against substantial departures from the preceding year’s outcome. It would appear that the future is often assumed to be much like the present.

ADO and Consensus Forecasts

Since the Consensus Forecasts represent judgments of a broader sample of economic forecasters, they generally reflect a wider mix of methods and sources of information than ADO forecasts. Although the Asia Pacific Consensus Forecasts began in 1995, a shorter common sample period is used, focusing on the post-Asian crisis years (i.e., 1999–2005). Significant turbulence during the Asian crisis years impaired forecasts.

Table 2 presents the summary statistics for the current-year and the next-year forecasts for GDP growth and inflation of both ADO and Consensus Forecasts.

GDP growth

GDP growth forecasts generally improved in the post-Asian crisis period. Both the MAEs and the median forecast errors are smaller in the shorter sample than in the full sample period. The MAE and median forecast error of ADO are also smaller than Consensus Forecasts. But the general pattern of underpredicting GDP growth remains. The current-year median forecast errors are all positively skewed and around 80% of current-year GDP forecasts are overpredictions. GDP growth forecast errors for the PRC and Viet Nam are all positive.

Again, both ADO and Consensus Forecasts show Malaysia, Thailand, and PRC as having the largest MAEs. Among the 9 DMCs examined here, Malaysia and Thailand recorded the highest change in GDP growth in 1999. From contraction of 7.4% in 1998, Malaysia posted growth of 6.1% in 1999. Similarly, Thailand’s GDP performance improved from a contraction of 10.5% in 1998 to 4.4% growth in 1999. And both ADO and Consensus Forecasts failed to foresee these sharp changes. The case of the PRC is different. Annual changes in GDP growth for 1999–2005 were small—less than 1 percentage point, yet both largely underpredicted PRC’s GDP growth.

The same broad patterns can be detected in next-year forecasts.

Inflation

Inflation errors are preponderantly negative. Again the ADO errors are slightly better than the Consensus Forecasts. For both of them, the mean absolute error for Viet Nam is more than 3 percentage points. This is primarily because of the large overprediction of inflation in 2000 when both ADO and Consensus Forecasts predicted positive inflation and the country experienced deflation. Absolute mean errors for Indonesia (in the case of ADO) and Pakistan (in the case of Consensus Forecasts) are also high—more than 2 percentage points. In the case of Indonesia, inflation was underpredicted in 1999 and 2005 by 3.4 and 4.6 percentage points, respectively. For Pakistan, inflation was, on the average, overpredicted by 2.3 percentage points, from 1999-2003 and underpredicted in 2005 by 3.1 percentage points.

Next-year inflation errors are higher than current-year. Mean absolute errors averaged more than 2 percentage points for both ADO and Consensus Forecasts. Five out of the nine DMCs have inflation errors of more than 2 percentage points, with Indonesia and Viet Nam having inflation errors higher than 4 percentage points.

The median forecast errors and the fraction of positive errors suggest a general tendency to overpredict inflation. It is only for India and Indonesia (in the case of ADO) that more than half the forecast errors are positive. The predominance of countries with proportions of positive signs below 0.5 is consistent with the tendency of forecasters to overpredict this year’s and next year’s inflation.

Conclusion

The analysis suggests that while GDP has generally been underpredicted inflation has been overpredicted. However, these errors are similar to—indeed, the forecasts are somewhat more accurate than—wider “industry” performance. Work is ongoing to recognize possible sources of errors and to improve forecasts.

References

Consensus Economics Inc. Various issues. Asia Pacific Consensus Forecasts.

Asian Development Bank. Various issues. Asian Development Outlook. Manila.

Pindyck, Robert and Daniel Rubinfeld. 1998. Econometric Models and Economic Forecasts. Singapore: McGraw-Hill Book Company.

Timmerman, Allan. 2006. “An Evaluation of the World Economic Outlook Forecasts.” IMF WP/06/59. International Monetary Fund, Washington, DC. March.

---

Back Statistical notes and tables	Asian Development Outlook 2006 Update