Derrick Kanngiesser and Tim Willems
This post describes a systematic way for central banks to employ past forecasts (and associated errors) with the aim of learning more about the structure and functioning of the economy, ultimately to enable a better setting of monetary policy going forward. Results suggest that the Monetary Policy Committee’s (MPC’s) inflation forecast has tended to underestimate pass-through from wage growth to inflation, while also underestimating the longer-term disinflationary impact of higher unemployment. Regarding the effects of monetary policy, our findings suggest that transmission through inflation expectations has played a bigger role than attributed to it in the forecast.
A sequence of unprecedented global shocks has recently posed a major challenge to economic forecasters across the world. Resulting forecast errors, particularly on inflation, have put central bank forecasts in the spotlight (see Bernanke (2024)).
The Bank of England MPC’s forecasts are constructed by drawing on a range of models, as described in Burgess et al (2013), augmented by staff and committee judgement. This raises questions of whether and how underlying forecast processes have contributed to forecast errors. In this regard, this post (which is based on our accompanying Staff Working Paper) proposes a strategy to use past Bank forecast errors to learn more about the UK economy. Generally speaking, out-of-sample forecasts are a good way to test the underlying forecast-generating model, which is what our approach leverages.
The data
At the heart of our exercise lies a data set of the Bank of England MPC’s quarterly forecasts from 2011 Q4 until 2024 Q1 for CPI inflation, wage growth, and real GDP growth (all annual rates, calculated year on year) as well as for the unemployment rate. In addition, we also use the yield on three-year UK government bonds, to analyse the impact of financial conditions (as shaped by UK monetary policy). We start our sample in 2011 Q4, as that date marks the adoption of the Bank’s current forecasting platform (Burgess et al (2013)).
Chart 1 plots the outturns for the year-on-year CPI inflation rate (in dark blue), while the light blue lines depict MPC forecasts made at various points in time. From late-2021 onward, one can observe repeated upside surprises as inflation continued to rise; the disinflation process – which started late-2022 – is, thus far, developing more in line with forecasts.
Chart 1: UK CPI inflation, outturns and modal forecasts
Forecast accuracy
A first test that we can consider is whether the MPC’s forecast have systematically under or overestimated key variables. That is, in statistical parlance, whether there is sign of any ‘bias’ in forecast errors.
Chart 2 shows the mean (blue line) and median (red line) forecast errors for inflation, wage growth, unemployment and GDP growth. We have defined the h quarter-ahead forecast error for variable 
The median forecast errors on inflation have been very close to zero across all horizons (mean errors – which are more sensitive to outliers – point towards an underprediction of inflation). That is to say that, despite recent forecast errors (stemming from having underestimated the pick-up in inflation following Russia’s invasion of Ukraine), there is no evidence of a systematic bias in the inflation forecast over a longer sample. At the same time, the medians in Chart 2 also show that the MPC’s forecast has tended to overpredict wage growth, unemployment and GDP growth.
Chart 2: Average forecast errors over 2011 Q4–2024 Q1
How forecasts can be leveraged to learn more about economic relationships
A key concept in the forecast evaluation literature is that of ‘forecast efficiency’. It implies that the forecast appropriately uses all information that was available to the forecaster at the time the forecast was made. A testable implication is that forecast errors should not be predictable using information available to the forecaster at the time the forecast was produced. Otherwise, the forecaster could have made a more accurate forecast by using that information.
An important observation in this regard was made by Blanchard and Leigh (2013), who noted that one potentially relevant piece of information available to the forecaster are forecasts of other variables. Based on this insight, they devised a strategy to see whether forecasters over or underestimate the strength of certain relationships within the economy. If a (correctly forecasted) movement in a certain driving variable (say, wage growth) is systematically associated with higher-than-forecasted inflation two years later, then the MPC’s forecast can be said to underestimate the impact of wage growth on inflation at the two-year horizon.
Our objective is to test whether the MPC’s forecast systematically over or underestimates the strength of the relationships between certain driving variables and inflation. We therefore regress forecast errors ( which is the forecast error on variable y at time t, based on the forecast made h quarters ago) on two-quarter ahead MPC forecasts of variable x (, but results are robust to other horizons):
Here, x represents, alternatively, the unemployment rate, wage growth, real GDP growth, or the three-year yield on UK government bonds. We estimate (1) using the method of ‘robust regression‘, which down-weights observations that are considered ‘atypical’ (in terms of the regression not producing a good fit to the data, for example during the Covid-period).
When looking at inflation outturns on the left-hand side of (1), a negative estimate for 
The black solid lines in Charts 3 and 4 show the estimates of for each horizon h equal to 1, 2, 4, 8 or 12; the shaded areas represent 90% confidence bands.
Chart 3: Blanchard-Leigh results for inflation forecast errors
Chart 3A shows that the estimated coefficient is negative at the two and three-year horizon. This means that forecasted wage growth is followed by inflation outturns that are higher than forecasted at the two and three-year forecasting horizon (where the grey confidence bands are different from zero). This suggests that MPC forecasts have underestimated the link from wage growth to inflation at those medium-term horizons.
Along similar lines, Chart 3B suggests that, at the two and three-year horizon, greater unemployment tend to be followed by lower-than-forecasted inflation outturns. This suggests that increases in unemployment might do more to lower inflation than historically encapsulated within the MPC’s forecast, which is for example consistent with the actual Phillips curve being steeper than encapsulated in the forecasting process.
According to Chart 3C, forecasted increases in GDP growth give rise to inflationary surprises up to horizons of two years, followed by lower-than-forecasted inflation at the three-year horizon. This could be due to the prospect of strong growth leading to some demand-driven inflation in the short run (more than forecasted), which is compensated by lower-than-forecasted inflation in later years (eg, thanks to increased supply arriving on the market). Alternatively, it could also be the case that policy reacts (more than was anticipated in the forecast) to the growth acceleration, responding in a way that tends to lower inflation (eg, tightening the fiscal or monetary policy stance).
Chart 4 reports equivalent findings for forecasted changes in three-year government bond yields. Since the latter are shaped by monetary policy, this exercise gives us an idea as to whether MPC forecasts have worked with an appropriate view of the monetary transmission mechanism. In this regard, it should be noted that the MPC forecast is not based on the MPC’s own forecast regarding the future path of interest rates, but is instead conditional on market-based interest rate forecasts. Results suggest that, relative to what has been encapsulated by the forecast, higher interest rates: A) have a stronger disinflationary impact over all horizons; B) push up unemployment by less at the three-year horizon; C) do less to slow real GDP growth at the two-quarter horizon; and D) do more to slow wage growth at the two and three-year horizon. Jointly, these observations are consistent with transmission through medium-term inflation expectations having played a bigger role than attributed to it in the forecast (as the inflation expectations channel can reduce inflation and wage growth without having to rely on a significant increase in unemployment; see Burr and Willems (2024)).
Chart 4: Blanchard-Leigh results for the monetary transmission mechanism
Conclusion
In this post, we have laid out a strategy through which central bank forecasts can be used to learn more about relationships between key variables of interest to the setting of monetary policy.
Our analysis suggests that there is scope for improvement by correcting certain relationships between variables within the MPC’s forecast. In particular, the pass-through from wage growth to inflation may be higher than assumed, while forecasts appear to have underestimated the longer-term disinflationary impact of higher unemployment. Finally, results suggest that monetary policy transmission via inflation expectations has played a bigger role than attributed to it in the forecasting process.
We hope these findings will be helpful in informing future forecast approaches, enabling improved setting of monetary policy.
Derrick Kanngiesser works in the Bank’s Monetary Policy Outlook Division and Tim Willems works in the Bank’s Structural Economics Division.
If you want to get in touch, please email us at [email protected] or leave a comment below.
Comments will only appear once approved by a moderator, and are only published where a full name is supplied. Bank Underground is a blog for Bank of England staff to share views that challenge – or support – prevailing policy orthodoxies. The views expressed here are those of the authors, and are not necessarily those of the Bank of England, or its policy committees.
Share the post “Forecast accuracy and efficiency at the Bank of England – and how forecast errors can be leveraged to do better”
Publisher: Source link
