Is predicting vol better worth the effort and does the VIX help?

 I'm a vol scaler.

There I've said it. Yes I adjust my position size inversely to vol. And so should you. 

But to this well we need to be able to predict future vol; where the 'future' here is roughly how long we expect to hold our positions for. 

Some people spend a lot of effort on this. They use implied vol from options, high(er) frequency data, GARCH or stochastic vol models. Other people don't spend a lot of effort on this. They look at the vol from the last month or so, and use that. I'm somewhere in the middle (though biased massively towards simplicity); I use an exponentially weighted moving average of recent vol combined with a much slower average.

An obvious question with any research effort is this: is the extra effort worth it? If we were trading options, then sure it would be. But we're not.

In this post I answer that 'is it worth spending time on this nonsense' question and look at the actual improvements we can gain from moving from the most rudimentary vol forecasting to the slightly more complex stuff I do. I also see if we can use a simple indicator of future volatility - the VIX - to improve things further. This was suggested by someone on Twitter(X). 

Is it worth predicting vol better?

I've mentioned this experiement a few times in the past, but I don't think I have ever blogged about it. Basically you run two backtests, one with your normal historic vol estimation, and the other with perfect foresight: basically equal to the ex-post vol over the next 30 days. This will be equal to the theoretical best possible job we could do if we really worked hard at forecasting vol. We can't do any better than a crystal ball. 

Then you check out the improvement. If vol is worth forecasting, there will be a big improvement in performance.

[This is a 'workhorse' test simulation with 100 liquid futures and 4 signals: 40% carry, and 20% in eahc of ewmac 16,32 and 64]

We begin with the simplest possible predictor of vol, a backward looking standard deviation estimate with an infinite window. Essentially this is a fixed vol estimate without any in sample estimation issues. We then compare that to the perfect foresight model.

Let's begin by looking and seeing what the vol outcome is like, this is one month rolling vol estimate (the realised vol of the strategy returns); clearly foresight does a better job of vol targeting.

Above are the cumulated returns. That sure looks like a decent improvement and as the vol of perfect foresight is lower it's better than it looks. It's a half unit improvement in SR points, from 0.76 to 1.24. The skew has dropped off from over 1.0 monthly to 0.12, but you know from my previous posts that small dip in skew won't be enough to destroy the huge CAGR advantage given by this sort of SR premium. The sortino is much better, more than double. 

So the short answer is yes, it's worth predicting vol better. Let's see how 

What size window

The obvious thing to do is to shorten our estimation window from forever to something a little shorter. Here is a graph I like to show people:

The x-axis shows the window size for a historic vol estimator in business days. The y-axis shows the R squared regressing the realised vol for a given future time period against the estimator / predictor of future vol. We're looking for the point on the x-axis that maximises R squared. Each line is a different length of future time period. So for example, to get the best prediction of vol one month ahead (about 21 business days) we look at the purple line for 21 days, and we can see this peaks at around 25 days. 

This is also the highest R squared. We are best at predicting one month vol ahead than other periods, and to do so we should use the previous one month vol (actually slightly more than a month). 

We don't do quite as well predicting shorter periods, and it looks like we might need slightly less data to predict eg 5 day vol. We do worse predicting longer periods, and it looks like we need more data. For 365 days ahead vol, the best R squared is obtained at somewhere between 40 days (around 2 months) and 100 days (around 5 months). 

Note: these are good R squared! In my last post a monthly holding period with an R squared of 0.1 would give us a SR of over 1, which is good. Here we are seeing R squared of over 0.30, which equates to a SR of nearly 2. That is very good - if we were as good at predicting returns as vol our SR would be two!

With that in mind, let's go from an infinite lookback to a 25 day business day lookback and see what happens.

First the rolling vol:

We can already see a fair improvement from the spikiness of the benchmark. How about the returns?

It looks like we are doing better than the benchmark and are competitive with foresight. However some of this is higher vol; our SR is 1.03 which still falls short of the 1.24 of the perfect foresight model, though obviously much better than the benchmark of infinite vol.

To recap:

Infinite previous vol                  SR 0.76

One month simple rolling vol           SR 1.03

Perfect foresight                      SR 1.24

From simple to exponential moving average

Now let's be a little fancier and go to EWM of vol rather than a simple equally weighted measure. This might not get us a better forecast of vol, but we should be smoother. A 36 day span in the pandas EWM function has the same half life as a 25 day SMA.

As before, here's the vol targeting, which is now almost identical:

And for profits....

Again we aren't quite vol matched, but EWM does in fact add a small increment in SR of 0.04 units. Around a quarter of that modest bump comes from lower costs (a saving of around 24 bp a year). 

Infinite previous vol                  SR 0.76

One month simple rolling vol           SR 1.03

One month EWM rolling vol              SR 1.06

Perfect foresight                      SR 1.24

I already looked at this in my book AFTS, but if we combine the standard 25 EWM vol with a very long run average (10 years) of the same vol we get another small bump. This is the vol measure I use myself.

Introducing the VIX

We are still some way short of getting close to perfect foresight vol. So let's do something else, for fun. We know that implied vol should be a good predictor of future vol; accounting for the well known vol premium (we get paid for being short gamma, hence implied is persistently higher than expected future vol).

Here's the simple rolling 25 day standard deviation measure for the S&P 500, and the VIX:

A couple of things to notice. Firstly the vol premium is larger after 2008 due to a general level of scaredy-cat-ness, then vanishes... and after around 2022 is noticeably negative. Over the last few years there have been a lot of dumb retail people selling vol and pushing the price down! The responsiveness to VIX of vol spikes has also become extremely muted.

Secondly it looks like VIX tracks rather than predicts increases in risk, at least for those unexpected events which cause the biggest spikes. Which suggests it's predictive power will be somewhat limited.

If we regress future vol on historic vol plus the VIX, the VIX coefficient is not significant; suggesting it isn't adding much to the party.

On the whole it doesn't look like VIX will be much help in predicting the future direction of vol, at least not without a great deal of work fitting time varying and probably non linear models.

Summary

There are significant performance benefits to be gained from forecasting vol well even in a directional system that doesn't trade optionality. Over half of those benefits can be captured by just using the right amount of lookback on a simple historical estimate. Further complexity can probably improve vol targeting but is unlikely to lead to significant performance improvements. Finally, the VIX is not especially helpful in predicting future volatility; this isn't helped by the variance premium flipping in the last few years.