Should I run my trading system at a fixed expected volatility target?

 This is a blog post which has been coming for a while. It relates to a lot of themes I've discussed before, and a recurring conversation I've had with a few people.

As most regular readers will know, I run my trading strategy to hit a particular risk target. That risk target is expressed as an annual standard deviation of percentage returns, and happens to be 25%. But those details aren't that important here. What is important is that it is a long run average risk target. Over the long run the expected risk in my backtest should be (pretty close to) 25% on average. If I can do a reasonable job of forecasting risk (and I usually can), then the actual ex-post standard deviation of returns in my backtest should also be 25% on average.

The important word here is on average. On a given day, my actual expected risk may be quite different from 25%. As I've discussed before, this is for two reasons. The first is something that I'm quite keen on: forecasting. A higher forecast means we have higher conviction in our trades, and thus our expected risk should be higher. The second thing is more of an annoyance: the relative correlation factor. It reflects the fact that my system is calibrated to size positions based on average historic correlations between market returns and positions. But on any given day these could be quite different, resulting in radically different expected risk.

There is no law saying you have to construct a trading strategy in this way. You could, for example, adjust your positions so that your expected risk is constant. Indeed this is common amongst long/short equity neutral funds which have historically tended to follow the classic Fama/French style factor model. Typically they go long the top quintile of their stock universe and go short the bottom quintile*. There is only a coarse link between forecast and position size here; everything is eithier in the portfolio as a long or short, or not in the portfolio at all (more formally, the relationship between forecast and position size is a thresholded binary mapping). The expected risk of this portfolio will depend only on todays covariance matrix. It makes perfect sense then to adjust the sizes of the resulting positions to target a fixed risk level. 

* I've clearly simplified things a lot here; most funds will adjust positions to hit a zero net Beta and may also neutralise sector and/or certain factor exposure.

Could we then apply such an approach to a system like mine that does use forecasts? Clearly, the main disadvantage of this approach is that you lose any information provided by the forecasts. In a period when forecasts were generally low, we'd gear up our positions to hit the target risk, and the converse would be true when forecasts were low. But that might not matter: if our portfolio is diversified enough then the range of dispersion in aggregate forecasts might not be that big. And another benefit could plausibly be an improvement in the characteristics of the portfolio. After all, vol targeting makes sense on a position level, so why not a portfolio level?

In this post I'll explore the idea of fixing ex-ante portfolio risk, and also propose a possible 'best of both worlds' compromise to this binary argument.

As I've already noted this post links to several previous posts:

• It's part three of a four(?) part series on forecasting, which started with this post. In that post I showed that forecasting mostly 'works' for individual instruments, with the caveat that there was some non linearity in the response of risk adjusted expected returns to forecasts (part of which can be explained by biased vol forecasting).
• In this older post I looked at volatility targeting on a position sizing level. Not quite the same, but interesting nonetheless. I found that vol targeting improved Sharpe Ratio and kurtosis, but reduced positive skew. I'll be looking at higher moment effects in this post as well.
• I'll draw heavily on this post I did on risk management to explain how forecasts influence expected risk, and where I also showed to calculate portfolio risk in pysystemtrade.

Why does expected risk vary?

This section is a direct copy from this post I did on risk management, so feel free to skip it if you remember something I wrote 5 whole months ago (I don't!).

The expected risk of my portfolio todaywill be wSw', where w are the current weights (basically position as % of capital) and S is my current estimate of the covariance matrix composed of instrument standard deviations and the correlation between instrument returns.

The position measured as a percentage of capital is a product of a lot of different numbers, but it simplifies to this:

position as % of capital = (instrument forecast / average instrument forecast)* (target risk / instrument risk) * instrument weight * IDM

Where the IDM (Instrument Diversification Multiplier) is the factor applied to positions to account for the correlation between trading subsystems (i.e. the trading strategies we run for each instrument and the returns they product, not the underlying instrument returns we use for S).

In a very handwaving way, it can be shown that the current expected portfolio risk will then be equal to:

Expected risk = target risk * (relative forecast strength) * (relative correlation factor)

Relative forecast strength is a measure of how strong aggregate forecasts are relative to the average; it is equal to the absolute forecasts for each instrument, weighted by instrument weight and divided by the average forecast (set to 10 in pysystemtrade).

All other things being equal, if your forecasts are all +20, and the average is +10, then your expected portfolio risk would be twice the average risk, or roughly twice the target risk (50% in the example I've been using).

This assumes that we want risk to vary according to aggregate forecast strength. Otherwise we'd have exactly the same risk on even if our forecasts were all +0.001, as if they were +20 (the maximum allowed under forecast capping). I'll check this assumption later in the post.

The relative correlation factor (RCF) is a bit more complicated. It is equal to the ratio between the IDM (which accounts for the average correlation across subsystem returns), and the IDM that would be appropriate today given the current set of positions and current correlation between instrument returns.

So for example, if you normally trade two subsystems (say US 10 year and S&P 500) with corelation between subsystems of zero then your IDM will be equal to square root of 2: 1.414

Now imagine that for some reason your system has a long average sized position in US 10 years, and a short average sized position in S&P 500 futures, and also that the correlation between these two instruments is -1. A quick calculation shows that the expected risk here will be 2.82 times the average. If the correlation was zero, then the expected risk would be twice the average; and if the correlation was +1 then the expected risk would be zero. The relevant RCF would be 2.82/1.41, 2/1.41, and zero.

Clearly the RCF can vary quite a lot depending on what the current positions are, and the current correlation matrix. You might argue that positions and correlations of this kind are unlikely given the average correlation between subsystems is zero. They are unlikely, but they aren't impossible. In particular, correlations do vary especially in the kind of market conditions we saw in March 2020.

The RCF is more of an annoyance in terms of expected risk; we wouldn't neccessarily want our risk to be a lot higher just because the positions we happen to have on are especially toxic given what todays correlations just happen to be.

... and if you've skipped, the rest of this post is original material.

Fixing portfolio risk

So we have a problem: the RCF means that our expected risk will move around quite a bit, regardless of our forecasts. And we have a possible solution, which is to fix the ex-ante expected portfolio risk. 

It's trivial to do this. Firstly we measure the expected risk of our portoflio Sp using the bolierplate portfolio risk calculation which is wSw', where w are the weights (basically position as % of capital) and S is the covariance matrix composed of instrument standard deviations and the correlation between instrument returns (different from that used for IDM).

We then calculate a risk adjustment factor, f = St / Sp where St is our risk target. Finally we multiply all our positions by f.

And this solution itself creates another problem! As I said above:

Expected risk = target risk * (relative forecast strength) * (relative correlation factor)

This means that f will be equal to:

f = 1/ [(relative forecast strength) * (relative correlation factor)]

Our adjusted expected risk, after applying f, will be:

Expected risk = target risk 

We've dealt with the dirty bathwater which the RCF is floating in, but we've also thrown out the baby that is the relative forecast strength. Is this a bad idea, or not?

How variable are forecasts at an aggregate level?

If relative forecast strength is a constant value of 1, then the only thing entering into our risk adjustment will be the RCF:

f = 1/ relative correlation factor

This is good! However, how realistic is this assumption? We know it's unrealistic for a system with a single instrument, since the forecast will usually range between -20 and +20. But what about for a portfolio of instruments? If the portfolio was sufficiently diversified, might changes in forecasts cancel each other out enough to produce an aggregate that was pretty much constant?

Let's check out the range of relative forecasts, using code from my previous post:

def forecast__strength_for_system(system):
    list_of_instruments = system.get_instrument_list()
    forecasts = [system.combForecast.get_combined_forecast(instrument_code)
              for instrument_code in
                list_of_instruments]
    forecasts = pd.concat(forecasts, axis=1)
    forecasts.columns = list_of_instruments
    forecasts = forecasts / system.config.average_absolute_forecast
    instrument_weights = system.portfolio.get_instrument_weights()

    weighted_forecast = instrument_weights.ffill() * forecasts.abs().ffill()
    forecast_strength = weighted_forecast.sum(axis=1)

    return forecast_strength

Aggregate absolute forecast weighted by instrument weights

That's certainly quite a range! Even after 2013, once we all 20 instruments in the portfolio, the aggregate measure of forecast strength ranges between less than 0.2 and above 1.3; a sixfold change in magnitude.

Do forecasts have forecasting power at an aggregate level?

In this post I spent quite a bit of time plotting ex-post risk adjusted return versus ex-ante forecast value, in an attempt to prove that forecast strength was indeed an indicator of future performance. 

Let's test this out at the aggregate level.

I'm reusing the 'bin plotting' functions from the previous post but I've dropped them into this gist for your edification. I've tweaked them slightly since the aggregate forecast can't be negative, so it makes sense to centre the plot on the mean forecast rather than zero.

def get_future_portfolio_return(Ndays):
    acc_curve = system.accounts.portfolio()
    acc_curve_sum = acc_curve.cumsum()
    period_returns = acc_curve_sum - acc_curve_sum.shift(Ndays)

    # We apply a single risk adjustment 

    stdev = acc_curve.std()

    scaled_returns_vol = stdev * (Ndays**.5)
    normalised_return = period_returns / scaled_returns_vol

    future_normalised_return = normalised_return.shift(-Ndays)

    return future_normalised_return

Ndays = 30
future_norm_return = get_future_portfolio_return(Ndays)
agg_forecast = forecast_strength_for_system(system)
pd_result = pd.DataFrame(dict(x=agg_forecast, y = future_norm_return))

plot_results_for_bin_size(6, pd_result, centre_on_mean=True)

Y-axis: risk adjusted ex-post portfolio return. X-axis: 

If fixed risk targets made sense and aggregate forecast levels were unimportant, then this would be a horizontal line. But it isn't; although it shows the weird noise we've come to expect from plots like this, it's definitely upward sloping. Bigger aggregate forecasts today mean higher expected returns in the future.

What effect does imposing a fixed risk target have on portfolio returns?

Let's implement a fixed risk target and see what effect it has on our portfolio returns. 

acc_curve = system.accounts.portfolio()

risk_series = get_expected_risk_for_system(system)
risk_series = risk_series.ffill()

# zero risk is bad because infinity
risk_series[risk_series==0]=np.nan
risk_vs_target = 100*risk_series / system.config.percentage_vol_target
risk_multiplier = 1/risk_vs_target

# let's not get carried away here guys
risk_multiplier[risk_multiplier>3.0] = 3.0

acc_curve_with_risk_multiplier = acc_curve * risk_multiplier

acc_curve.cumsum().plot()

acc_curve_with_risk_multiplier.cumsum().plot()

Again, the code for get_expected_risk is from a previous post, but here it is in a gist to avoid you having to dig it out (I'm feeling super generous today). Note that because we're just multiplying all positions by a fixed scalar, we see the results of this approximately just by multiplying the portfolio returns for the underlying system, rather than implementing some fancy function inside the backtesting engine.

This does mean that costs might be slightly wrong, since one system will have more trading than the other.

Red line: Account curve without fixed risk targeting. Blue curve: Account curve with fixed risk targeting

As we might expect from our findings so far, fixed risk targeting is not an unmitigated success. The Sharpe Ratio drops from a princely 0.93 to a measly 0.64; a gap big enough to be statistically significant. The only light in this especially dark tunnel is that the underperformance isn't quite as bad in the last couuple of decades.

But remember that vol targeting on individual instruments affects both SR but also higher moments. What about skew and kurtosis? Meh. Skew is actually worse with fixed risk targeting (at daily, weekly, and monthly horizons becoming significantly more negative), but on the bright side kurtosis is better. The 1% tail of returns is much worse with fixed risk targets at a daily frequency, but pretty similar for monthly and weekly returns. 

Bear in mind that the parameter uncertainty for skew and kurtosis is quite high.

 

There must be some good news! How well does fixed vol targeting do at targeting vol? Let's plot the 3 month rolling vol estimate for both systems:

Y-axis: Rolling 6 month realised daily standard deviation of returns. Blue line: Original System. Orange line: Fixed risk target

The fixed risk system does what it says on the tin; mostly realised vol hovers around the target, with an exception in around 2013. So if predictable risk is more important to you than performance (or skew!), then by all means fix your risk target. 

Of course it's possible that a more sophisticated version of fixed risk targeting could be more successful here. We could for example run a Markowitz style optimisation, using the expected returns implied by our forecasts and standard deviation estimates, with a classical maximum risk constraint (that we'd end up hitting, since we'd be maximising return). 

However that would be a radical departure for the way this kind of trading system worked. If it was successful, we wouldn't know whether it was the risk targeting that was good, or the optimisation that was good. 

I'm not a big fan of optimisation (understatement!) so this isn't something I'd want to try in practice anyway.

Can we use forecasts but account for changes in correlation and positions?
Let's return again to this equation:

Expected risk = target risk * (relative forecast strength) * (relative correlation factor)

What we really want is something that does something like this:

Expected risk = target risk * (relative forecast strength)

We've tried targeting a fixed risk, which means multiplying all our positions by f: 

f= St / Sp 

(where St is our risk target and Sp is our portfolio risk)

f = 1/ [(relative forecast strength) * (relative correlation factor)]
And that didn't go well. But what if we multiplied our positions by f*, which is f multiplied by the relative forecast strength:

f= (relative forecast strength)*St / Sp 

f* = (relative forecast strength)/ [(relative forecast strength) * (relative correlation factor)]

 = 1 / (relative correlation factor)]

Now we're cooking with some kind of petroleum vapour! f* only corrects for the relative correlation factor, leaving forecast strength to be unaffected.

What does this look like in code:

risk_multiplier_forecasts = (agg_forecast/agg_forecast.mean())*risk_multiplier
acc_curve_with_forecast_risk_multiplier = acc_curve * risk_multiplier_forecasts

Since the aggregate forecast measure isn't guaranteed to have a mean of 1, we account for this.

Red line: Account curve without fixed risk targeting. Blue curve: Account curve with fixed risk targeting, accounting for forecast magnitude

Now we have something! Cumulatively on returns alone the new risk targeting actually beats the raw system, if only by a little bit. The Sharpe Ratio is actually a little bit lower than originally: 0.87 rather than 0.93, but Sharpe Ratio isn't everything and that isn't a significant difference. 

Daily skew is a little worse than before, but both weekly and monthly skew are better with the new target. On the downside, kurtosis is higher using f* than for the original system; and the 1% tail is also worse.

Conclusion

The simpler fixed risk target idea, where we target 25% annualised risk or whatever every single day, is a non-starter. It dramatically reduces performance with no clear benefit to other characteristics of the strategy, apart from the more predictable realised risk. And the reason why is clear; it throws away the information provided by aggregate forecast strength, which is a clear predictor of future risk adjusted return.

A more complicated risk target which takes aggregate forecast strength into account looks better, with performance that's on par with the original system depending on what metric you look at. But I probably wouldn't implement this in my trading system as it is, it's not a clear improvement in returns and there is a substantial increase in complexity. 

It isn't clear whether this makes the system safer; yes there will be times when the positions we have on and the current correlations coincide to create much higher ex-ante risk than we would normally want; something the fixed risk target deals with (although the risk model discussed here also does this at least for extremes). 

But there will be other times when our position appears to be ex-ante low risk because of where correlations are, and the risk targeting will leverage up, but then correlations break down in a crisis and we suddenly have much higher ex-post risk than we expected.

However, the idea of targeting a fixed risk level with an adjustment for aggregate forecast strength is quite a neat one and it may make sense for other types of trading systems. Feel free to steal it!

This is part three of a four part series on forecasting. Parts one and two are available. Part four is yet to be written.