Department of Economics Økonomisk institutt


Background to economic forecasting

Introduction

As humans we can't help making predictions and forecasts. For example, when we find a joke funny, isn't it because we are surprised by the "point" of the story, having unwittingly anticipated a different ending of the story? Hence, forecast errors are in this case a source of fun, while they are often a drag when they impinge on everyday life. For example: exam questions don't always come out as you anticipated during your revisions, the weekend weather doesn't always develop as the wheatherman says, and your football club's expensive new striker may turn out to be a flop.

In economics, forecasts errors are often a source of embarrassment, a pattern of reaction which in part is due to having too high expectations about how accurate economic forecasts can be made, realistically speaking. Building on a new theory about economic forecasting, see e.g., Hendry (2001), these web-pages take as a starting point that, in principle, forecast failures are unavoidable in economics. However, approaches exist which help reduce the incidence rate of forecast failures, and which reduces the damages inflicted by failures when they nevertheless occur.

There is a range of sources for forecast errors in economics, see below. As long as the joint effect of these factors amount to a stochastic error-term with a stable distribution, informative measures of forecast uncertainty can be calculated, and the forecast can be presented as probabilistic forecasts, which is more useful than merely presenting a forecast number (e.g., 5% rate "unemployment next year"). The most common way of presenting forecast uncertainty is by forecast confidence intervals. Such intervals are expected to cover the likely outcomes some percentage of the time, such as 67% , 90% or 95%.

Measures of forecast uncertainty have multiple uses. First, the probabilistic forecasts convey to the public that the forecasted numbers (for GDP, inflation etc) only will coincide with future realizations on average, and the confidence intervals show the whole range of likely and less likely outcomes. Second, experience tells us that forecast errors turn out to to be larger, and more systematic, than what the uncertainty measures allow for. In other words, realizations which the forecasts depict as highly unlikely (e.g., outside the computed confidence interval) have a tendency to materialize too often. This then, is the defining characteristic of forecast failure, and it also reminds us that the conventional uncertainty measures only reflect a proportion of the full uncertainty about future outcomes. According to the modern theory of forecasting, the main source of large forecast errors and forecast failures are intermittent structural breaks in the parameters governing growth rates of economic variables, and the means of equilibrium combinations of economic variables.

In the face of frequent but irregular regime shifts, there is a premium on having a robust and adaptable forecasting process. Allowing relevant historical shocks to be reflected in forecast uncertainty calculations contributes to robustness in the forecasts, else the reported forecast confidence intervals may be become too narrow, giving rise to a false impression of a high degree of precision in the forecasts. This is an argument for having a presence of econometric modeling in the forecasting process. Correctly interpreted and used, an econometric forecasting model also equips the forecaster with a tool that gives her a chance of adapting reasonably quick to regime shifts that "have moved" from the forecasting period to the historical sample period that conditions the forecasts, thus avoiding unnecessary spells of bad forecasting. "Correct use" entails that the econometric forecasting model is used in conjunction with other (secondary) forecasting methods, for example time series models that are highly adaptable to regime shifts.

Methods of forecasting

A forecast is a statement about the future, obtained by collection, interpretation and (sometimes) formal analysis of historical experience and data. Interpretation presumes a conceptual framework, so no forecasts is 'without theory'. The different practical forecasting methods only differ on the type of theory they have as their reference framework. Pure guessing aside, all methods also make use of quantitative data analysis, but the type of technology used of course differs a great deal.

Methods of forecasting include

Each of these approaches are used in practice, and although some forecasting institutes specialize in one method (for example the Ifo World Economic Survey), most professional forecasters use a combination of methods in their forecasting process.

Econometric and time series models are the primary methods of forecasting in economics. The time dimension is essential in forecasting, so methods that give full attention to the temporal properties of the data stand a chance of producing reasonable forecasts, whereas models with no (or with only stylized) dynamics are unsuited for forecasting. For some time , in the 1960s and 1970s, this expained why statistical time series models had the cutting edge over econometric models in forecast comparisons, see Granger and Newbold (1986). In the 1980s, macroeconometric models took on board many of the methodological aspects of time series analysis, and important developments also took place within the econometric discipline itself. Today, it goes without saying that extensive empirical evaluation shouild be an integral part of the model building process.

As a result, modern econometric forecasting models are less exposed to Granger's and Newbold's critique. At the same time, 'pure' time series models remain highly adaptable to the kind of regime shifts that typically will produce forecast failure in econometric models, if allowed to remain unrepresented in those systems. Hence, a combination of the two methods are recommended.

Econometric forecasting models

The econometric approch to forecasting consists of several stages, for example:

Though, other methods of forecasting, in particular time series models using differenced data, are even more adaptable, econometric models can be made to adapt to breaks that have occurred prior to making the forecasts. Econometric models are also attractive since, to a larger extent than other methods, they can account for their own failures, and it is possible to learn form past mistakes, see e.g., Eitrheim, Jansen and Nymoen (2002). Finally, by using an econometric model, we can illustrate interesting alternative scenarios of the future, corresponding to alternative assumption of the future paths taken by policy instrument variables, or of other exogenous variables .

Forecast confidence intervals

As stated in the introduction, the most common way of presenting forecast uncertainty is by forecast confidence intervals. Such intervals are expected to cover the likely outcomes some percentage of the time, such as 67% , 90% or 95%.

A popular graphical representation of forecast confidence intervals is the fan-charts. In these graph, the line of 'point forecast' is drawn as the most probable forecast, while ranges of intervals in ever lighter shades of colour (or of grey) fan out above and below as the likelihood of outcomes outside each bound falls.

It is important to note that reported intervals are based on 'known uncertainties', which will only cover part of total uncertainty in store. Hence, even if correctly computed, for the best available forecasting method, forecast confidence intervals tend to downplay forecast uncertainty. By the same token, forecasters will do worse than they expect from their forecast intervals. One frequent source of realizations outside the forecast confidence regions are structural breaks that takes place in the economy, after the preparation and publication of the forecasts. Below we refer to this as post-forecast structural breaks.

Forecast failure

Forecast failures occur when forecasts errors are excessively large compared to what is provided for by the confidence regions, or when there is a sequence of bad forecasts. The later variant is particularly worrying, since it can be a sign of too little adaptability with respect to regime shifts in the forecasting process.

All (known) methods of forecasting are subject to forecast failure from time to time. Hence, the origins of forecast failures are important to understand within the 'rationality' of each methods. These pages are concerned with time series and econometric methods, and the account for their intermittent failures given in e.g. Clements and Hendry (1999) and popularized in Hendry (2001).

Econometric forecasting models capture relationships between variables, such as inflation, unemployment, interest rates and foreign currency exchange rates. The relationships are written as 'systems of equations' are then estimated from available data which are subject to measurement errors and subsequent revisions. The models can be said to have three main components, see Hendry (2001):

As explained in Hendry (2001), the key to understanding systematic forecast failure lies in the behaviour of the deterministic terms. In particular, unanticipated changes in the values of the deterministic terms harm the forecasts, and is often the source of forecast failure. Other sources of forecast errors, such as mis-specifying the stochastic components, or uncertainty due to parameter estimation appear to be less important as 'stand alone' factors causing persistently bad forecasts. However, when these problems interact with regimes shifts (unanticipated changes in deterministic terms) their pernicious effects are increased. For example, an important practical problem is that forecasts necessarily are conditioned on preliminary data that are less accurate than the finally revised data. In conjunction with a structural break, using a a wrong set of initial conditions can lead to forecast failure.

For discussion of a recent forecast failure, affecting Norwegian inflation, see Nymoen (2005)..

Literature

Clements, M.P and D.F. Hendry. (1999). Forecasting Non-stationary Economic Time Series, MIT Press. Cambridge Massachusetts.
Hendry, D.F. (2001). How Economists Forecast. Ch 1 in Understanding Economic Forecasting, D.F. Hendry and N.E. Ericsson (eds). MIT Press. Cambridge Massachusetts.
Granger, C.W.J. and P. Newbold (1986). Forecasting Economic Time Series. Academic Press, San Diego.

This document maintained by {ragnar.nymoen@econ.uio.no}.
Material Copyright © 2001.