Fossicking in the dark or twenty-twenty foresight? – Bank Underground

Rishi Khiroya and Lydia Henning

If you asked people what skill they would most love to have, you might receive answers like ‘to fly’, ‘to be invisible’ or even ‘predicting the future’. If you asked people who worked in financial markets in particular, ‘accurately predicting the future’ would probably be top of the list. From economic trends to political shifts, market participants have a stake in anticipating what comes next. We use data collected from the Bank’s Market Participants Survey (MaPS) to see how market predictions have tended to compare with what subsequently unfolds over the period of high uncertainty and volatility that has been observed in the wake of the pandemic – and how predictive accuracy has varied depending on the time horizon in question.

The MaPS is a survey of expectations for monetary policy run two weeks prior to every Monetary Policy Committee (MPC) meeting to gather information on topics relevant to the MPC. The MaPS started as a pilot in mid-2020 before being formally launched in February 2022, with the results published on the Bank’s website 24 hours after each MPC decision (see Andrea Rosen’s speech).

Now that we’ve set the scene, we begin by looking at the very near-term outlook for policy – in particular the level of Bank Rate that transpires from the most immediately approaching MPC meeting. In Chart 1, the purple line plots the median ‘most likely’ Bank Rate expectation at each MaPS survey (ie the median expectation for September 2024 MPC recorded in the September 2024 MaPS, the median expectation for the November 2024 MPC recorded in the November 2024 MaPS etc) while the dotted white line plots realised Bank Rate.

Chart 1: Realised Bank Rate against median expectations for the upcoming meeting

We can see that the median market participant has correctly predicted what would happen to Bank Rate at the next meeting for 19 out of the 22 meetings covered in the sample to date.

How does this hold up when we extend the prediction window?

Chart 2 shows the average share of respondents whose Bank Rate projections recorded one, two and three policy announcements prior to the policy announcement in question have subsequently been realised. The average share for those who predicted the outcome of the prevailing survey meeting is shown in deep purple for comparison. As you would expect, the closer the market is to a decision, the more accurate their prediction tends to be, as information is revealed and incorporated into expectations.

Chart 2: Predictive accuracy through the cycle

Looking through the sample, as you might expect, there was a higher tendency for predictions to be realised during periods where Bank Rate was being held constant than when it was on the move.

What if we extend further out again?

Chart 3: Realised Bank Rate against median profile recorded

Chart 3 compares the median expected profile for Bank Rate over the following 12 months – recorded at different points through the recent cycle – with the realised path. From this we can see that, up until when Bank Rate was reaching its peak, market participants tended to undershoot how high rates would go. Interestingly though, in September 2023 the median prediction was for a slightly higher peak than what was realised.

Chart 4 compares subsequent realisations against median MaPS predictions out to the one-year horizon. When the criteria is set as an exact match, such a ‘hit’ was observed 19% of the time over the MaPS sample. However, when we allow for a 25 basis points threshold either side of realised Bank Rate, the average accuracy was 40%.

Chart 4: Average accuracy of median Bank Rate expectations

Another alternative and more lenient benchmark considers only the direction of the path for Bank Rate – in other words, does it go up, down or stay the same. By this measure (the darkest orange bar), we see the median expected path for Bank Rate tends to evolve in the same direction as what is realised around 60% of the time.

Finally, we also see some evidence of predictive accuracy varying over our sample. It is evident that the proportion of subsequently realised median predictions increased through early 2023 in midst of MPC’s tightening cycle, before ebbing as Bank Rate reached its peak and increasing again into the subsequent holding period. This could be consistent with respondents ‘learning’ as they become accustomed to the cycle and adapting their expectations accordingly.

And we can’t talk about Bank Rate without talking about its other half – inflation.

Chart 5: Average absolute deviation from realised inflation prints

In Chart 5, we use the same approach as for Bank Rate and compare median MaPS expectations with subsequent realisations on inflation. The results display a similar (and expected pattern) with the average deviation being lowest at the nearest horizon at which we ask for expectations before trailing off.

Chart 6: Average absolute deviation in predictions through the cycle

As we can see in Chart 6, we also observe material variability across the sample. The pattern is more monotonic than is the case with Bank Rate with the gap between predictions (out to the one-year horizon) and realisations narrowing through the time period. Splitting inflation expectations by calendar year of when the MaPS occurred, MaPS respondents’ average absolute deviation from realised prints has decreased by around 3.5 times between 2022 and 2024 – with respondents adapting to the upward spike in inflation, followed by the subsequent decline and relative levelling out.

In Chart 7 we plot the median MaPS expected profile for inflation at various points during this cycle and realised inflation in white. Similar to Bank Rate, the median profile tended to undershoot what subsequently realised, up until realised inflation reached its peak. Further through the sample, as markets recalibrated, their expectations moved closer to realised inflation.

Chart 7: Realised inflation against median profile recorded

At the outset we posed the question ‘fossicking in the dark or twenty-twenty foresight?’. The evidence from the MaPS (unsurprisingly) shows that neither applies definitively, with somewhere in between being a more representative characterisation. It must be said though that the time window encompassed by our sample comprises some periods of unprecedented volatility which needs to be highlighted on the report card – along with the observation that market participants appeared to adapt to their evolving environments and ‘learn’.

Rishi Khiroya and Lydia Henning work in the Bank’s Market Intelligence and Analysis Division.

If you want to get in touch, please email us at [email protected] or leave a comment below.

Comments will only appear once approved by a moderator, and are only published where a full name is supplied. Bank Underground is a blog for Bank of England staff to share views that challenge – or support – prevailing policy orthodoxies. The views expressed here are those of the authors, and are not necessarily those of the Bank of England, or its policy committees.