Introducing max-GM (median@p), a new(?) performance statistic

Do you remember this post? https://qoppac.blogspot.com/2022/06/vol-targeting-cagr-race.html

Here I introduced a performance metric, the best annualised compounding return at the optimal leverage level for that strategy. This is equivalent to finding the highest geometric return once a strategy is run at it's Kelly optimal leverage.

I've since played with that idea a bit, for example in this more recent post amongst others I considered the implications of that if we have different tolerances for uncertainty and used bootstrapping of returns when optimising a stock and bond portfolio with leverage, whilst this post from last month does the same exercise in a bitcoin/equity portfolio without leverage.

In this post I return to the idea of focusing on this performance metric - the maximised geometric mean at optimal leverage, but now I'm going to take a more abstract view to try and get a feel for in general what sort of strategies are likely to be favoured when we use this performance metric. In particular I'd like to return to the theme of the original post, which is the effect that skew and other return moments have on the maximum achievable geometric mean. 

Obvious implications here are in comparing different types of strategies, such as trend following versus convergent strategies like relative value or option selling; or even 'classic' trend following versus other kinds.

Some high school maths

(Note I use 'high school' in honor of my US readers, and 'maths' for my British fans)

To kick off let's make the very heroric assumption of Gaussian returns, and assume we're working at the 'maximum Kelly' point which means we want to maximise the median of our final wealth distribution - same as aiming for maximum geometric return - and are indifferent to how much risk such a portfolio might generate.

Let's start with the easy case where we assume the risk free rate is zero; which also implies we pay no interest for borrowing (apologies I've just copied and pasted in great gobs of LaTeX output since it's easier than inserting each formula manually):

Now that is a very nice intuitive result!

Trivially then if we can use as much leverage as we like, and we are happy to run at full Kelly, and our returns are Gaussian, then we should choose the strategy with the highest Sharpe Ratio. What's more if we can double our Sharpe Ratio, we will quadruple our Geometric mean!

Truely the Sharpe Ratio is the one ratio to rule them all!

Now let's throw in the risk free rate:

We have the original term from before (although the SR now deducts the risk free rate), and we add on the risk free rate reflecting the fact that the SR deducts it, so we add it back on again to get our total return. Note that the higher SR = higher geometric mean at optimal leverage relationship is still true. Even with a positive risk free rate we still want to choose the strategy with the highest Sharpe Ratio!

Consider for example a classic CTA type strategy with 10% annualised return and 20% standard deviation, a miserly SR of 0.5 with no risk free rate; and contrast with a relative value fixed income strategy that earns 6.5% annualised return with 6% standard deviation, a SR of 1.0833

Now if the risk free rate is zero we would prefer the second strategy as it has a higher SR, and indeed should return a much higher geometric mean (since the SR is more than double, it should be over four times higher). Let's check. The optimal leverages are 2.5 times and 18.1 (!) times respectively. At those leverages the arithmetic means are 25% and 117% respectively, and the geometric means using the approximation are 12.5% for the CTA and 58.7% for the RV strategy.

But what if the risk free rate was 5%? Our Sharpe ratios are now equal: both are 0.25. The optimal leverages are also lower, 1.25 and 4.17. The arithmetic means come in at 12.5% and 27.1%, with geometric means of 9.4% and 24%. However we have to include the cost of interest; which is just 1.25% for the CTA strategy (borrowing just a quarter of it's capital at a cost of 5% remember) but a massive 15.8% for the RV. Factoring those in the net geometric means drop to 8.125% for both strategies - we should be indifferent between them, which makes sense as they have equal SR.

The horror of higher moments

Now there is a lot wrong with this analysis. We'll put aside the uncertainty around being able to measure exactly what the Sharpe Ratio of a strategy is likely to be (which I can deal with by drawing off more conservative points of the return distribution, as I have done in several previous posts), and the assumption that returns will be the same in the future. But that still leaves us with the big problem that returns are not Gaussian! In particular a CTA strategy is likely to have positive skew, whilst an RV variant is more likely to be a negatively skewed beast, both with fat tails in excess of what a Gaussian model would deliver. In truth in the stylised example above I'd much prefer to run the CTA strategy rather than a quadruple leveraged RV strategy with horrible left tail properties.

Big negative skewed strategies tend to have worse one day losses; or crap VAR if you prefer that particular measure. The downside of using high leverage is that we will be saddled with a large loss on a day when we have high leverage, which will significantly reduce our geometric mean.

There are known ways to deal with modifying the geometric mean calculation to deal with higher moments like skew and kurtosis. But my aim in this post is to keep things simple; and I'd also rather not use the actual estimate of kurtosis from historic data since it has large sampling error and may underestimate the true horror of a bad day that can happen in the future (the so called 'peso problem'); I also don't find the figures for kurtosis particuarly intuitive. 

(Note that I did consider briefly using maximum drawdown as my idiots tail effect here. However maximum drawdown is only relevant if we can't reduce our leverage into the drawdown. And perhaps counter intuitively, negative skewed strategies actually have lower and shorter drawdown)

Instead let's turn to the tail ratio, which I defined in my latest book AFTS. A lower tail ratio of 1.0 means that that the left tail is Gaussian in size, whilst a higher ratio implies a fatter left tail.

I'm going to struggle to rewrite the relatively simple 0.5SR^2 formulae to include a skew and left tail term, which in case will require me to make some distributional assumptions. Instead I'm going to use some bootstrapping to generate some distributions, measure the tail properties, find the optimal leverage, and then work out the geometric return at the optimal leverage point. We can then plot maximal geometric means against empirical tail ratios to get a feel for what sort of effect these have.

Setup

To generate distributions with different tail properties I will use a mixture of two Gaussian distributions; including one tail distribution with a different mean and standard deviation* which we draw from with some probability<0.5. It will then be straightforward to adjust the first two moments of the sample distribution of returns to equalise Sharpe Ratio so we are comparing like with like.

* you will recognise this as the 'normal/bliss' regime approach used in the paper I discussed in my prior post around optimal crypto allocation, although of course it will only be bliss if the tail is nicer which won't be the case half the time.

As a starting point then my main return distribution will have daily standard deviation 1% and mean 0.04% which gives me an annualised SR of 0.64, and will be 2500 observations (about 10 years) in length - running with different numbers won't affect things very much. For each sample I will draw the probability of a tail distribution from a uniform distribution between 1% and 20%, and the tail distribution daily mean from uniform [-5%, 5%], and for the tail standard deviation I will use 3% (three times the normal). 

All this is just to give me a series of different return distributions with varying skew and tail properties. I can then ex-post adjust the first two moments so I'm hitting them dead on, so the mean, standard deviation and SR are identical for all my sample runs. The target standard deviation is 16% a year, and the target SR is 0.64, all of which means that if the returns are Gaussian we'd get a maximum leverage of 4.0 times.

As always with these things it's probably easier to look at code, which is here (just vanilla python only requirements are pandas/numpy).

Results

Let's start with looking at optimal leverage. Firstly, how does this vary with skew?

Ignore the 'smearing' effect this is just because each of the dots in a given x-plane will have the same skew and first two moments, but slightly different distribution otherwise. As we'd expect given the setup of the problem the optimal leverage with zero skew is 4.0

For a pretty nasty skew of -3 the leverage should be about 10% lower - 3.6; whilst for seriously positive skew of +3 you could go up to 4.5. These aren't big changes! Especially when you consider that few real world trading strategies have absolute skew values over 1 with the possible exception of some naked option buying/selling madness. The most extreme skew value I could find in my latest book was just over 2.0, and that was for single instruments trading carry in just one asset class (metals).

What about the lower tail? I've truncated the plot on the x-axis for reasons I will explain in a second.

Again for the base case with a tail ratio of 1.0 (Gaussian) the optimal leverage is 4.0; for very thin lower tails again it goes up to 4.4, but for very fat lower tails it doesn't really get below about 3.6. Once again, tail ratios of over 4.0 are pretty rare in real strategies (though not individual instruments), although my sampling does sometimes generate very high tail ratios the optimal leverages never go below 3.6 even for a ratio of nearly 100.

Upper tail, again truncated:

Again a tail ratio of 1.0 corresponds to optimal leverage of roughly 4.0; and once again even for very fat upper tail ratios (of the sort that 'classical' trend followers like to boast about, the optimal leverage really isn't very much higher.

Now let's turn to Geometric mean, first as affected by skew:

As we'd expect we can generate more geometric mean with more skew... but not a lot! Even a 3 unit improvement in skew barely moves the return needle from 22.5% to 24.5%. To repeat myself it's rare to find trading strategies above or below 1.0, never mind 3.0. For example the much vaunted Mulvaney capital has a return skew on monthly returns of about 0.45. The graph above shows that will only add about 0.25% to expected geometric return at optimal leverage versus a Gaussian normal return distribution (this particular fund has extremely large standard deviation as they are one of the few funds in the world that actually runs at optimal leverage levels).

For completeness, here are the tail ratios. First the lower tail ratio:

Fat left tails do indeed reduce maximum optinmal geometric means, but not a lot.

Now for the upper tail:

Summary

I do like neat formulae, and the results for Gaussian normal distributions do have the property of being nice. Of course they are based on unreal expectations, and anyway who runs full Kelly on a real strategy expecting the returns to be normal and the SR to be equal to the backtested SR is an idiot. A very optimistic idiot, no doubt with a sunny disposition, but an idiot nonetheless.

For the results later in the post it does seem surprising that even if you have something with a very ugly distribution that you'd not really adjust your optimal leverage much, and hence see barely on impact on geometric mean. But remember here that I'm fixing the first two moments of the distribution; which means I'm effectively assuming that I can measure these with certainty and also that the future will be exactly like the past. These are not realistic expectations! 

And that is why in the past rather than choose the leverage that maximised geometric mean, I've chosen to maximise some more conservative point on the distribution of terminal wealth (where geometric mean would be the median of that distribution). Doing that would cause some damage to the negative skewed, fat left tail distributions resulting in lower optimal leverage and thus lower geometric means.

I have thought of a way of doing this analysis with distributional points, but it's quite computationally intensive and this post is already on the long side, so let's stop here. 

Portfolio optimisation, uncertainty, bootstrapping, and some pretty plots. Ho, ho, ho.

Optional Christmas themed introduction

Twas the night before Christmas, and all through the house.... OK I can't be bothered. It was quiet, ok? Not a creature was stirring... literally nothing was moving basically. And then a fat guy in a red suit squeezed through the chimney, which is basically breaking and entering, and found a small child waiting for him (I know it sounds dodgy, but let's assume that Santa has been DBS checked*, you would hope so given that he spends the rest of December in close proximity to kids in shopping centres)

* Non british people reading this blog, I could explain this joke to you, but if you care that much you'd probably care enough to google it.

"Ho ho" said the fat dude "Have you been a good boy / girl?"

"Indeed I have" said the child, somewhat precociously if you ask me.

"And what do you want for Christmas? A new bike? A doll? I haven't got any Barbies left, but I do have a Robert Oppenheimer action figure; look if you pull this string in his stomach he says 'Now I am become Death destroyer of worlds', and I'll even throw in a Richard Feynman lego mini-figure complete with his own bongo drums if you want."

"Not for me, thank you. But it has been quite a long time since Rob Carver posted something on his blog. I was hoping you could persuade him to write a new post."

"Er... I've got a copy of his latest book if that helps" said Santa, rummaging around in his sack "Quite a few copies actually. Clearly the publisher was slightly optimistic with the first print run."

"Already got it for my birthday when it came out in April" said the child, rolling their eyes.

"Right OK. Well I will see what I can do. Any particular topic you want him to write about in this blog post?"

"Maybe something about portfolio optimisation and uncertainty? Perhaps some more of that bootstrapping stuff he was big on a while ago. And the Kelly criterion, that would be nice too."

"You don't ask for much, do you" sighed Santa ironically as he wrote down the list of demands.

"There need to be really pretty plots as well." added the child. 

"Pretty... plots. Got it. Right I'll be off then. Er.... I don't suppose your parents told you to leave out some nice whisky and a mince pie?"

"No they didn't. But you can have this carrot for Rudolf and a protein shake for yourself. Frankly you're overweight and you shouldn't be drunk if you're piloting a flying sled."

He spoke not a word, but went straight to his work,And filled all the stockings, then turned with a jerk. And laying his finger aside of his nose, And giving a nod, up the chimney he rose! He sprang to his sleigh, to his team gave a whistle, And away they all flew like the down of a thistle. But I heard him exclaim, ‘ere he drove out of sight,

"Not another flipping protein shake..."

https://pixlr.com/image-generator/ prompt: "Father Christmas as a quant trader"

Brief note on whether it is worth reading this

I've talked about these topics before, but there are some new insights, and I feel it's useful to combine the question of portfolio weights and optimal leverage into a single post / methodology. Basically there is some familar stuff here but now in a coherent story, plus some new stuff.

And there are some very nice plots.

Somewhat messy python code is available here (with some data here or use your own), and it has no dependency on my open source trading system pysystemtrade so everyone can enjoy it.

Bootstrapping

I am a big fan of bootstrapping. Some definitional stuff before I explain why. Let's consider a couple of different ways to estimate something given some data. Firstly we can use a closed form. If for example we want the average monthly arithmetic return for a portfolio, we can use the very simple formula of adding up the returns and dividing by the number of periods. We get a single number. Although the arithmetic mean doesn't need any assumptions, closed form formula often require some assumptions to be correct - like a Gaussian distribution. And the use of a single point estimate ignores the fact that any statistical estimate is uncertain. 

Secondly, we can bootstrap. To do this we sample the data repeatedly to create multiple new sets of data. Assuming we are interested in replicating the original data series, the new set of data would be the same length as the original, and we'd be sampling with replacement (or we'd just get the new data in a different order). So for example, with ten years of daily data (about 2500 observations), we'd choose some random day and get the returns data from that. Then we'd keep doing that, not being bothered about choosing the same day (sampling with replacement), until we had done this 2500 times. 

Then from this new set of data we estimate our mean, or do whatever it is we need to do. We then repeat this process, many times. Now instead of a single point estimate of the mean, we have a distribution of possible means, each drawn from a slightly different data series. This requires no assumptions to be made, and automatically tells us what the uncertainty of the parameter estimate is. We can also get a feel for how sensitive our estimate is to different variations on the same history. As we will see, this will also lead us to produce estimates that are more robust to the future being not exactly like the past.

Note: daily sampling destroys any autocorrelation properties in the data, so it wouldn't be appropriate for example for creating new price series when testing momentum strategies. To do this, we'd have to sample larger chunks of time period to retain the autocorrelation properties. For example we might restrict ourselves to sampling entire years of data. For the purposes of this post we don't mind about autocorrelation, so we can sample daily data.

Bootstrapping is particularly potent in the field of financial data because we only have one set of data: history. We can't run experiments to get more data. Bootstrapping allows us to create 'alternative histories' that have the same basic character as our actual history, but aren't quite the same. Apart from generating completely random data (which itself will still require some assumptions - see the following note), there isn't really much else we can do.

Bootstrapping helps us with the quant finance dilemma: we want the future to be like the past so that we can use models calibrated on the past in the future, but the future will never be exactly like the past. 

Note: that bootstrapping isn't quite the same as monte carlo. With that we estimate some parameters for the data, making an assumption about it's distribution. Then we randomly sample from that distribution. I'm not a fan of this. We have all the problems of making assumptions about distribution, and of uncertainty about the parameter estimates we use for that distribution. 

Portfolio optimisation

With all that in mind, let's turn to the problem of portfolio opimisation. We can think as this as making two decisions:

• Allocating weights to each asset, where the weights sum to one
• Deciding on the total leverage for the portfolio

Under certain assumptions we can seperate out these two decisions, and indeed this is the insight of the standard mean variance framework and the 'security market line'. The assumption is that enough leverage is available that we can get to the risk target for the investor. If the investor has a very low risk tolerance, we might not even need leverage, as the optimal portfolio will consist of cash + securities.

So basically we choose the combination of asset weights that maximises our Sharpe Ratio, and then we apply leverage to hit the optimal risk target (since SR is invariant to leverage, that will remain optimal). 

To begin with I will assume we can proceed in this two phase approach; but later in the post I will relax this and look at the effect of jointly allocating weights and leverage.

I'm going to use data for S&P 500 and 10 year Bond futures from 1982 onwards, but which I've tweaked slightly to produce more realistic forward looking estimates for means and standard deviations (in fact I've used figures from this report- their figures are actually for global equities and bonds, but this is all just an illustration). 

My assumptions are:

• Zero correlation (about what it has been in practice since 1982)
• 2.5% risk free rate (which as in standard finance I assume I can borrow at)
• 3.5% bond returns @ 5% vol
• 5.75% equity returns @ 17% vol

This is quite a nice technique, since it basically allows us to use forward looking estimates for the first two moments (and first co-moment - correlation) of the distribution, whilst using actual data for the higher moments (skew, kurtosis and so on) and co-moments (co-skew, co-kurtosis etc). In a sense it's sort of a blend of a parameterised monte-carlo and a non parameterised bootstrap.

Optimal leverage and Kelly

I'm going to start with the question of optimal leverage. This may seem backwards, but optimal leverage is the simpler of the two questions. Just for illustrative purposes, I'm going to assume that the allocation in this section is fixed at the classic 60% (equity), 40% (bonds). This gives us vol of around 10.4% a year, a mean of 4.85%, and a Sharpe Ratio of 0.226

The closed form solution for optimal leverage which I've written about at some length, is the Kelly Criterion. Kelly will maximise E(log(final wealth)) or median(final wealth), or importantly here it will maximise the geometric mean of your returns.

Under the assumption of i.i.d. Gaussian returns optimal Kelly leverage is achieved by setting your risk target as an annual standard deviation equal to your Sharpe Ratio. With a SR of 0.226 we want to get risk of 22.6% a year, which implies running at leverage of 22.6 / 10.4 = 2.173

That of course is a closed form solution, and it assumes that:

• Return parameters are Guassian i.i.d. (which financial data famously is not!)
• The return parameters are fixed
• That we have no sampling uncertainty of the return parameters
• We are fine running at fully Kelly, which is a notoriously aggressive amount of leverage

Basically that single figure - 2.173 - tells us nothing about how sensitive we would be to the future being similar to, but not exactly like, the past. For that we need - yes - bootstrapping. 

Bootstrapping optimal leverage 

Here is the bootstrap of my underlying 60/40 portfolio with leverage of 1.

Each point on this histogram represents a single bootstrapped set of data, the same length as the original. The x-axis shows the geometric mean, which is what we are trying to maximise. You can see that the mean of this distribution is about 4.1%. Remember the arithmetic mean of the original data was 4.85%, and if we use an approximation for geometric mean that assumes Gaussian returns then we'd get 4.31%. The difference between 4.1% and 4.31% is because this isn't Guassian. In fact, mainly thanks to the contribution of equities, it's left tailed and also has fat tails. Left fat tails result in lower Geometric returns - and hence also a lower optimal leverage, but we'll get to that in a second.

Notice also that there is a fair bit of distributional range here of the geometric mean. 10% of the returns are below 2%, and 1% are below 0.4%.

Now of course I can do this for any leverage level, here it is for leverage 2:

The mean here is higher, as we'd probably expect since we know the optimal leverage would be just over 2.0 if this was Gaussian. It comes in at 4.8%; versus the 7.2% we'd expect if this was the arithmetic mean, or the 5.04% that we would have for Gaussian returns.

Now we can do something fun. Repeating this exercise for many different levels of leverage, we can take each of the histograms that are producing and pull various distributional points off them. We can take the median of each distribution (50% percentile, which in fact is usually very close to the mean), but also more optimistic points such as the 75% and 90% percentile which would apply if you were a very optimistic person (like SBF, as I discussed in a post about a year ago), and perhaps more usefully the 25% and 10% points. We can then plot these:

How can we use this? Well, first of all we need to decide what our tolerance for uncertainty is. What point on the distribution are you optimising for? Are you the sort of person who worries about the bad thing that will happen 1 in 10 times, or would you instead be happy to go with the outcome that happens half the time (the median)?

This is not the same as your risk tolerance! In fact, I'm assuming that your tolerance for risk is sufficient to take on the optimal amount of leverage implied by this figure. Of course it's likely that someone with a low tolerance for risk in the form of high expected standard deviation would also have a low tolerance for uncertainty. And as we shall see, the lower your tolerance for uncertainty, the lower the standard deviation will be on your portfolio.

(One of the reasons I like this framing of tolerance is that most people cannot articulate what they would consider to be an appropriate standard deviation, but most people can probably articulate what their tolerance for uncertainty is, once you have explained to them what it means)

Next you should focus on the relevant coloured line, and mentally remove the odd bumps that are due to the random nature of bootstrapping (we could smooth them out by using really large bootstrap runs - note they will be worse with higher leverage since we get more dispersion of outcomes based on one or two bad days eithier being absent or repeated in the sample), and then find the optimium leverage.

For the median this is indeed at roughly the 2.1 level that theory predicts (in fact we'd expect it to be a little lower because of the negative skew), but this is not true of all the lines. For inveterate gamblers at 90% it looks like the optimum is over 3, whilst for those who are more averse to bad outcomes at 10% and 25% it's less than 2; in fact at 10% it looks like the optimium could easily be 1 - no leverage. These translate to standard deviations targets of somewhere around 10% for the person with a 10% risk tolerance . 

Technical note: I can of course use corrections to the closed form Kelly criterion for non Gaussian returns, but this doesn't solve the problem of parameter estimation uncertainty - if anything it makes it worse.

The final step, and this is something you cannot do with a closed form solution, is to see how sensitive the shape of the line is to different levels of leverage, thus encouraging us to go for a more robust solution that is less likely to be problematic if the future isn't exactly like the past. Take a slightly conservative 25% quantile person on the red line in the figure. Their optimium could plausibly be at around 1.75 leverage if we had a smoother plot, but you can see that there is almost no loss in geometric mean from using less leverage than this. On the other hand there is a steep fall off in geometric mean once we get above 1.75 (this assymetry is a property of the geometric mean and leverage). This implies that the most robust and conservative solution would be to choose an optimal leverage which is a bit below 1.75. You don't get this kind of intuition with closed form solutions.

Optimal allocation - mean variance

Let's now take a step backwards to the first phase of solving this problem - coming up with the optimal set of weights summing to one. Because we assume we can use any amount of leverage, we want to optimise the Sharpe Ratio. This can be done in the vanilla mean-variance framework. The closed form solution for the data set we have, which assumes Gaussian returns and linear correlation, is a 22% weight in equities and 78% in bonds. That might seem imbalanced, but remember the different levels of risk. Accounting for this, the resulting risk weights are pretty much bang on 50% in each asset. 

As well as the problems we had with Kelly, we know that mean variance has a tendency to produce extreme and not robust outcomes, especially when correlations are high. If for example the correlation between bonds and equities was 0.65 rather than zero, then the optimal allocation would be 100% in bonds and nothing in equities.

(I actually use an optimiser rather than a single equation to calculate the result here, but in principal I could use an equation which would be trivial for two assets - see for example my ex colleague Tom's paper here - and not that hard for multiple assets eg see here).

So let's do the following; boostrap a set of return series with different allocations to equities (bond allocation just 100% - equity allocation), then measure the Sharpe Ratio of each allocation/bootstrapped return series, and then measure the distribution of those Sharpe Ratios for different distributional points.

Again, each of these coloured lines represents a different point on the distribution of Sharpe Ratios. The y-axis is the Sharpe Ratio, and the x-axis is the allocation to equities; zero in equities on the far left, and 100% on the far right. 
Same procedures as before: first work out your tolerance for uncertainty and hence which line you should be on. Secondly, find the allocation point which maximises Sharpe Ratio. Thirdly, examine the consequences of having a lower or higher allocation - basically how robust is your solution.
For example, for the median tolerance (green line) the best allocation comes in somewhere around 18%. That's a little less than the closed form solution; again this is because we haven't got normally distributed assets here. And there is a reasonably symettric shape to the gradient around this point, although that isn't true for lower risk tolerances.
You may be surprised to see that the maximum allocation is fairly invarient to uncertainty tolerance; if anything there seems to be a slightly lower allocation to equities the more optimistic one becomes (although we'd have to run a much more granular backtest plot to confirm this). Of course this wouldn't be the case if we were measuring arithmetic or even geometric return. But on the assumption of a seperable portfolio weighting problem, the most appropriate statistic is the Sharpe Ratio. 
This is good news for Old Skool CAPM enthusiasts! It really doesn't matter what your tolerance for uncertainty is, you should put about 18% of your cash weight - about 43% of your risk weight in equities; at least with the assumption that future returns have the forward looking expectations for means, standard deviations, and correlations I've specified above; and the historic higher moments and co-moments that we've seen for the last 40 years.

 

Joint allocation

Let's abandon the assumption that we can seperate our the problem, and instead jointly optimise the allocation and leverage. Once again the appropriate statistic will be the geometric return. We can't plot these on a single line graph, since we're optimising over two parameters (allocation to equities, and overall leverage), but what we can do is draw heatmaps; one for each point on the return distribution.
Here is the median:

The x-axis is the leverage; lowest on the left, highest on the right. The y-axis is the allocation to equities; 0% on the top, 100% on the bottom. And the heat colour on the z-axis shows the geometric return. Dark blue is very good. Dark red is very bad. The red circle shows the highest dark blue optimum point. It's 30% in equities with 4.5 times leverage: 5.8% geometric return.
But the next question we should be asking is about robustness. An awful lot of this plot is dark blue, so let's start by removing everything below 3% so we can see the optimal region more clearly:

You can now see that there is still quite a big area with a geometric return over 5%. It's also clear from the fact there is variation of colour within adjacent points that the bootstrapped samples are still producing enough randomness to make it unclear exactly where the optimium is; and this also means if we were to do some statistical testing we'd be unable to distinguish between the points that are whiteish or dark blue. 
In any case when we are unsure of the exact set of parameters to use, we should use a blend of them. There is a nice visual way of doing this. First of all, select the region you think the optimal parameters come from. In this case it would be the banana shaped region, with the bottom left tip of the banana somewhere around 2.5x leverage, 50% allocation to equities; and the top right tip around 6.5x leverage, 15% allocation. And then you want to choose a point which is safely within this shape, but further from steep 'drops' to much lower geometric returns which means in this case you'd be drawn to the top edge of the banana. This is analogous to avoiding the steep drop when you apply too much leverage in the 'optimal leverage' problem. 
I would argue that something around the 20% point in equities, leverage 3.0 is probably pretty good. This is pretty close to a 50% risk weight in equities, and the resulting expected standard deviation of 15.75% is a little under equities. In practice if you're going to use leverage you really should adjust your position size according to current risk, or you'd get badly burned if (when) bond vol or equity vol rises.
Let's look at another point on the distribution, just to get some intuition. Here is the 25% percentile point, again with lower returns taken out to better intuition:

The optimal here stands out quite clearly, and in fact it's the point I just chose as the one I'd use with the median! But clearly you can see that the centre of gravity of the 'banana' has moved up and left towards lower leverage and lower equity allocations, as you would expect. Following the process above we'd probably use something like a 20% equity allocation again, but probably with a lower leverage limit - perhaps 2.

Conclusion

Of course the point here isn't to advocate a specific blend of bonds and equities; the results here depend to some extent on the forward looking assumptions that I've made. But I do hope it has given you some insight into how bootstrapping can give us much more robust outcomes plus some great intuition about how uncertainty tolerance can be used as a replacement for the more abstract risk tolerance. 
Now go back to bed before your parents wake up!

If you're so smart, how come you're not SBF? The Kelly criterion and choice of expectations and utility function when bet sizing

There has been a very interesting discussion on twitter, relating to some stuff said by Sam Bankman-Fried (SBF), who at the time of writing has just completely vaporized billions of dollars in record time via the medium of his crypto exchange FTX, and provided a useful example to future school children of the meaning of the phrase nominative determinism*.

* Sam, Bank Man: Fried. Geddit? 

Read the whole thread from the top:

https://twitter.com/breakingthemark/status/1591114381508558849

TLDR the views of SBF can be summarised as follows:

• Kelly criterion maximises log utility
• I don't have a log utility function. It's probably closer to linear.
• Therefore I should bet at higher than Kelly. Up to 5x would be just fine.

I, and many others, have pointed out that SBF is an idiot. Of course it's easier to do this when he's just proven his business incompetence on a grand scale, but to be fair I was barely aware of the guy until a week ago. Specifically, he's wrong about the chain of reasoning above*. 

* It's unclear whether this is specifically what brought SBF down. At the time of writing he appears to have taken money from his exchange to prop up his hedge fund, so maybe the hedge fund was using >>> Kelly leverage, and this really is the case. 

In this post I will explain why he was wrong, with pictures. To be clearer, I'll discuss how the choice of expectation and utility function affects optimal bet sizing. 

I've discussed parts of this subject briefly before, but you don't need to read the previous post.

Scope and assumptions

To keep it tight, and relevant to finance, this post will ignore arguments seen on twitter related to one off bets, and whether you should bet differently if you are considering your contribution to society as a whole. These are mostly philosophical discussions which it's hard to solve with pictures. So the set up we have is:

• There is an arbitrary investment strategy, which I assume consists of a data generating process (DGP) producing Gaussian returns with a known mean and standard deviation (this ignores parameter uncertainty, which I've banged on about often enough, but effectively would result in even lower bet sizing).
• We make a decision as to how much of our capital we allocate to this strategy for an investment horizon of some arbitrary number of years, let's say ten.
• We're optimising L, the leverage factor, where L =1 would be full investment, 2 would be 100% leverage, 0.5 would be 50% in cash 50% in the strategy and so on.
• We're interested in maximising the expectation of f(terminal wealth) after ten years, where f is our utility function.
• Because we're measuring expectations, we generate a series of possible future outcomes based on the DGP and take the expectation over those.

Note that I'm using the contionous version of the Kelly criterion here, but the results would be equally valid for the sort of discrete bets that appear in the original discussion.

Specific parameters

Let's take a specific example. Set mean =10% and standard deviation = 20%, which is a Sharpe ratio of 0.5, and therefore Kelly should be maxed at 50% risk, equating to L = 50/20 = 2.5. SBF optimal leverage would be around 5 times that, L = 12.5. We start with wealth of 1 unit, and compound it over 10 years.

I don't normally paste huge chunks of code in these blog posts, but this is a fairly short chunk:

import pandas as pd
import numpy as np
from math import log

ann_return = 0.1
ann_std_dev = 0.2

BUSINESS_DAYS_IN_YEAR = 256
daily_return = ann_return / BUSINESS_DAYS_IN_YEAR
daily_std_dev = ann_std_dev / (BUSINESS_DAYS_IN_YEAR**.5)

years = 10
number_days = years * BUSINESS_DAYS_IN_YEAR


def get_series_of_final_account_values(monte_return_streams,
                                       leverage_factor = 1):
    account_values = [account_value_from_returns(returns,
                    leverage_factor=leverage_factor)
                      for returns in monte_return_streams]

    return account_values

def get_monte_return_streams():
    monte_return_streams = [get_return_stream() for __ in range(10000)]

    return monte_return_streams

def get_return_stream():
    return np.random.normal(daily_return,
                            daily_std_dev,
                            number_days)

def account_value_from_returns(returns, leverage_factor: float = 1.0):
    one_plus_return = np.array(
        [1+(return_item*leverage_factor)
         for return_item in returns])
    cum_return = one_plus_return.cumprod()

    return cum_return[-1]

monte_return_streams = get_monte_return_streams()

Utility function: Expected log(wealth) [Kelly]

Kelly first. We want to maximise the expected log final wealth:

def expected_log_value(monte_return_streams,
                       leverage_factor = 1):

    series_of_account_values =get_series_of_final_account_values(
        monte_return_streams = monte_return_streams,
        leverage_factor = leverage_factor)
    log_values_over_account_values = [log(account_value) for account_value in series_of_account_values]

    return np.mean(log_values_over_account_values)

And let's plot the results:

def plot_over_leverage(monte_return_streams, value_function):
    leverage_ratios = np.arange(1.5, 5.1, 0.1)
    values = []
    for leverage in leverage_ratios:
        print(leverage)
        values.append(
                value_function(monte_return_streams, leverage_factor=leverage)
        )

    leverage_to_plot = pd.Series(
        values, index = leverage_ratios
    )

    return leverage_to_plot

leverage_to_plot = plot_over_leverage(monte_return_streams,
                                      expected_log_value)
leverage_to_plot.plot()

In this plot, and nearly all of those to come, the x-axis shows the leverage L and the y-axis shows the value of the expected utility. To find the optimal L we look to see where the highest point of the utility curve is.

As we'd expect:

• Max expected log(wealth) is at L=2.5. This is the optimal Kelly leverage factor.
• At twice optimal we expect to have log wealth of zero, equivalent to making no money at all (since starting wealth is 1).
• Not plotted here, but at SBF leverage (12.5) we'd have expected log(wealth) of <undefined> and have lost pretty much all of our money.

Utility function: Expected (wealth) [SBF?]

Now let's look at a linear utility function, since SBF noted that his utility was 'roughly close to linear'. Here our utility is just equal to our terminal wealth, so it's purely linear.

def expected_value(monte_return_streams,
                       leverage_factor = 1):

    series_of_account_values =get_series_of_final_account_values(
        monte_return_streams = monte_return_streams,
        leverage_factor = leverage_factor)

    return np.mean(series_of_account_values)

leverage_to_plot = plot_over_leverage(monte_return_streams,
                                      expected_value)

You can see where SBF was coming from, right? Utility gets exponentially higher and higher, as we add more leverage. Five times leverage is a lot better than 2.5 times, the Kelly criterion. Five times Kelly, or 2.5 * 5= 12.5, would be even better.

Utility function: Median(wealth) 

However there is an important assumption above, which is the use of the mean for the expectation operator. This is dumb. It would mean (pun, sorry), for example, that of the following:

1. An investment that lost $1,000 99 times out of 100; and paid out $1,000,000 1% of the time
2. An investment that is guaranteed to gain $9,000 

... we would theoretically prefer option 1 since it has an expected value of $9,010, higher than the trivial expected value of $9,000 for option 2. There might be some degenerate gamblers who prefer 1 to 2, but not many.

(Your wealth would also affect which of these you would prefer. If $1,000 is a relatively trivial amount to you, you might prefer 1. If this is the case consider if you'd still prefer 1 to 2 if the figures were 1000 times larger, or a million times larger). 

I've discussed this before, but I think the median is the more appropriate. What the median implies in this context is something like this: 

Considering all possible future outcomes, how can I maximise the utility I receive in the outcome that will occur half the time?

I note that the median of option 1 above is zero, whilst the median of option 2 is $9,000. Option 2 is now far more attractive.

def median_value(monte_return_streams,
                       leverage_factor = 1):

    series_of_account_values =get_series_of_final_account_values(
        monte_return_streams = monte_return_streams,
        leverage_factor = leverage_factor)

    return np.median(series_of_account_values)

The spooky result here is that the optimal leverage is now 2.5, the same as the Kelly criterion.

Even with linear utility, if we use the median expectation, Kelly is the optimal strategy.

The reason why people prefer to use mean(log(wealth)) rather than median(wealth), even though they are equivalent, is that the former is more computationally attractive.

Note also the well known fact that Kelly also maximises the geometric return.

With Kelly we aren't really making any assumptions about utility function: our assumption is effectively that the median is the correct expectations operator

The entire discussion about utility is really a red herring. It's very hard to measure utility functions, and everyone probably does have a different one, I think it's much better to focus on expectations.

Utility function: Nth percentile(wealth) 

Well you might be thinking that SBF seems like a particularly optimistic kind of guy. He isn't interested in the median outcome (which is the 50% percentile). Surely there must be some percentile at which it makes sense to bet 5 times Kelly? Maybe he is interested in the 75% percentile outcome?

QUANTILE = .75
def value_quantile(monte_return_streams,
                       leverage_factor = 1):

    series_of_account_values =get_series_of_final_account_values(
        monte_return_streams = monte_return_streams,
        leverage_factor = leverage_factor)

    return np.quantile(series_of_account_values, QUANTILE)

Now the optimal is around L=3.5. This is considerably higher than the Kelly max of L=2.5, but it is still nowhere near the SBF optimal L of 12.5.

Let's plot the utility curves for a bunch of different quantile points:

list_over_quantiles = []
quantile_ranges = np.arange(.4, 0.91, .1)
for QUANTILE in quantile_ranges:
    leverage_to_plot = plot_over_leverage(monte_return_streams,
                                          value_quantile)
    list_over_quantiles.append(leverage_to_plot)

pd_list = pd.DataFrame(list_over_quantiles)
pd_list.index = quantile_ranges
pd_list.transpose().plot()

It's hard to see what's going on here, legend floating point representation notwithstanding, but you can hopefully see that the maximum L (hump of each curve) gets higher as we go up the quantile scale, as the curves themselves get higher (as you would expect).

But in none of these quantiles we are still nowhere near reaching an optimal L of 12.5. Even at the 90% quantile - evaluating something that only happens one in ten times - we have a maximum L of under 4.5.

Now there will be some quantile point at which L=12.5 is indeed optimal. Returning to my simple example:

1. An investment that lost $1,000 99 times out of 100; and paid out $1,000,000 1% of the time
2. An investment that is guaranteed to gain $9,000 

... if we focus on outcomes that will happen less than one in a million times (the 99.9999% quantile and above) then yes sure, we'd prefer option 1.

So at what quantile point does a leverage factor of 12.5 become optimal? I couldn't find out exactly, since to look at extremely rare quantile points requires very large numbers of outcomes*. I actually broke my laptop before I could work out what the quantile point was. 

* for example, if you want ten observations to accurately measure the quintile point, then for the 99.99% quantile you would need 10* (1/(1-0.9999)) = 100,000 outcomes.

But even for a quantile of 99.99% (!), we still aren't at an optimal leverage of 12.5! 

You can see that the optimal leverage is 8 (around 3.2 x Kelly), still way short of 12.5.

Summary

Rather than utility functions, I think it's easier to say to ask people the likelihood of outcome they are concerned about. I'd argue that sensible people would think about the median outcome, which is what you expect to happen 50% of the time. And if you are a bit risk averse, you should probably consider an even lower quantile. 

In contrast SBF went for bet sizing that would only make sense in the set of outcomes that happens significantly less than 0.01% of the time. That is insanely optimistic; and given he was dealing with billions of dollars of other peoples money it was also insanely irresponsible.

Was SBF really that recklessly optimistic, or dumb? In this particular case I'd argue the latter. He had a very superfical understanding of Kelly bet sizing, and because of that he thought he could ignore it. 

This is a classic example of 'a little knowledge is a dangerous thing'. A dumb person doesn't understand anything, but reads on the internet somewhere that half Kelly is the correct bet sizing. So they use it. A "smart" person like SBF glances at the Kelly formula, thinks 'oh but I don't have log utility' and leverages up five times Kelly and thinks 'Wow I am so smart look at all my money'. And that ended well...

A truely enlightened person understands that it isn't about the utility function, but about the expectation operator. They also understand about uncertainty, optimistic backtesting bias, and a whole bunch of factors that imply that even 0.5 x Kelly is a little reckless. I, for example, use something south of a quarter Kelly. 

Which brings us back to the meme at the start of the post:

Note I am not saying I am smarter than SBF. On pure IQ, I am almost certainly much, much dumber. In fact, it's because I know I am not a genius that I'm not arrogant enough to completely follow or ignore the Kelly criteria without first truely understanding it.

Whilst this particular misunderstanding might not have brought down SBF's empire, it shows that really really smart people can be really dumb - particularly when they think that they are so smart they don't need to properly understand something before ignoring it*.

* Here is another example of him getting something completely wrong

Postscript (16th November 2022)

I had some interesting feedback from Edwin Teejay on twitter, which is worth addressing here as well. Some of the feedback I've incorporated into the post already.

(Incidentally, Edwin is a disciple of Ergodic Economics, which has a lot of very interesting stuff to say about the entire problem of utility maximisation)

First he commented that the max(median) = max(log) relationship is only true for a long sequence of bets, i.e. asymptotically. We effectively have 5000 bets in our ten year return sequence. As I said originally, I framed this as a typical asset optimisation problem rather than a one off bet (or small number of one off bets).

He then gives an example of a one off bet decision where the median would be inappropriate:

1. 100% win $1
2. 51% win $0 / 49% win $1'000'000

The expected values (mean expectation) are $1 and $490,000 respectively, but the medians are $1 and $0. But any sane person would pick the second option.

My retort to this is essentially the same as before - this isn't something that could realistically happen in a long sequence of bets. Suppose we are presented with making the bet above every single week for 5 weeks. The distribution of wealth outcomes for option 1 is single peaked - we earn $5. The distribution of wealth outcomes for option 2 will vary from $0 (with probability 3.4%) to $5,000,000 (with a slightly lower probability of 2.8% - I am ignoring 'compounding', eg the possibility to buy more bets with money we've already won), with a mean of $2.45 million. 

But the median is pretty good: $1 million. So we'd definitely pick option 2. And that is with just 5 bets in the sequence. So the moment we are looking at any kind of repeating bet, the law of large numbers gets us closer and closer to the median being the optimal decision. We are just extremely unlikely to see the sort of payoff structure in the bet shown in a series of repeated bets.

Now what about the example I posted:

1. An investment that lost $1,000 99 times out of 100; and paid out $1,000,000 1% of the time
2. An investment that is guaranteed to gain $9,000 

Is it realistic to expect this kind of payoff structure in a series of repeated bets? Well consider instead the following:

1. An investment that lost $1 most of the time; and paid out $1,000,000 0.001% of the time
2. An investment that is guaranteed to gain $5

The mean of these bets is ~$9 and $5, and the medians are $-1 and $5.

Is this unrealistic? Well, these sorts of payoffs do exist in the world- they are called lottery tickets (albeit it is rare to get a lottery ticket with a $9 positive mean!). And this is something closer to the SBF example, since I noted that he would have to be looking at somewhere north of the 0.01% quantile to choose 5x Kelly Leverage.

Now what happens if we run the above as a series of 5000 repeated bets (again with no compounding for simplicity).  We end up with the following distributions:

1. An investment that lost $5000 95.1% of the time, and makes $1 million or more 5% of the time.
2. An investment that is guaranteed to gain $25,000

Since there is no compounding we can just multiply up the individual numbers to get the mean ($45,000  and $25,000 respectively). The medians are -$5,000 and $25,000. Personally, I still prefer option 2! You might still prefer option 2 if spending $5,000 on lottery tickets over 10 years reflects a small proportion of your wealth, but I refer you to the previous discussion on this topi: make 

So I would argue that it in a long run of bets we are more likely in real life to get payoff structures of the kind I posited, than the closer to 50:50 bet suggested by Edwin. Ultimately, I think we agree that for long sequences of bets the median makes more sense (with a caveat). I personally think long run decision making is more relevant to most people than one off bets. 

What is the caveat? Edwin also said that the choice of the median is 'arbitrary'. I disagree here. The median is 'what happens half the time'. I still think for most people that is a logical reference point for 'what I expect to happen', as well as in terms of the maths: both median and mean are averages after all. I personally think it's fine to be more conservative than this if you are risk averse, but not to be more aggressive - bear in mind that will mean you are betting at more than Kelly.

But anyway, as Matt Hollerbach, whose orginal series of tweets inspired this post, said:

"The best part of Robs framework is you don't have to use the median,50%. You could use 60%  or 70%  or 40% if your more conservative.  And it intuitively tells you what the chance of reaching your goal is. You don't get duped into a crazy long shot that the mean might be hiding in."  (typos corrected from original tweet)

This fits well into my general framework for thinking about uncertainty. Quantify it, and be aware of it. Then if you still do something crazy/stupid, well at least you know you're being an idiot...

Exogenous risk overlay: take two

This is a short follow up post to one I did a couple of years ago, on "Exogenous risk management". This was quite an interesting post which dug into why expected risk changes for a typical diversified futures trading system. And then I introduced my risk overlay:

"Now we have a better understanding of what is driving our expected risk, it's time to introduce the risk overlay. The risk overlay calculates a risk position multiplier, which is between 0 and 1. When this multiplier is one we make no changes to the positions calculated by our system. If it was 0.5, then we'd reduce our positions by half. And so on.

So the overlay acts across the entire portfolio, reducing risk proportionally on all positions at the same time. 

The risk overlay has three components, designed to deal with the following issues:

- Expected risk that is too high

- Weird correlation shocks combined with extreme positions

- Jumpy volatility (non stationary and non Gaussian vol)

Each component calculates it's own risk multipler, and then we take the lowest (most conservative) value.

That's it. I could easily make this a lot more complicated, but I wanted to keep the overlay pretty simple. It's also easy to apply this overlay to other strategies, as long as you know your portfolio weights and can estimate a covariance matrix "

Since I wrote that post, I've radically changed the way my trading strategy is designed. This is discussed at length here. Here are some relevant highlights

"Things that worked, but I decided not to implement:

• ....
• A risk overlay (actually was in my old system, but not yet in my new system - a change I plan to make soon)
• ....

Things that worked and I did implement during 2021:

• Dynamic optimisation using minimum tracking error (here and here)
• .....
• Estimating vol using a partially mean reverting method

Basically I did a lot of stuff, but along the way I lost my risk overlay. But like I said above, I planned to reintroduce it once the dust had settled. Can you hear that noise? That's sound the sound of dust settling. Also note that I introduced a new method for estimating vol. That will be important later.

There will be some very specific references to my open source trading system pysystemtrade, but the vast majority of this post should be of general interest to most people who are interested in risk management.

Where should the risk overlay go?

This was a question that wasn't an issue before, but now I have my new fancy dynamic optimisation method, where should the risk overlay go? Should it go into the upstream methodology that calculates an optimal unrounded set of positions? Or should it go into the actual dynamic optimisation itself? Or should it go into a downstream process that runs after the dynamic optimisation?

On reflection I decided not to put the overlay into the dynamic optimisation. You can imagine how this would have worked; we would have added the risk limits as additional constraints on our optimisation, rather than just multipling every positions by some factor betwen 0 and 1. It's a nice idea, and if I was doing a full blown optimisation it would make sense. But I'm using a 'greedy algo' for my optimisation, which doesn't exhaustively search the available space. That makes it unsuitable for risk based constraints. 

I also decided not to put the overlay into a downstream process. This was for a couple of reasons. Firstly, in production trading the dynamic optimisation is done seperately from the backtest, using the actual live production database to source current positions and position constraint + trade/no-trade information. I don't really want to add yet more logic to the post backtest trade generation process. Secondly - and more seriously - having gone to a lot of effort to get the optimal integer positions, applying a multiplier to these would most probably result in a lot of rounding going on, and the resulting set of contract positions would be far from optimal.

This leaves us with an upstream process; something that will change the value of system.portfolio.get_notional_positions(instrument_code) - my optimal unrounded positions. 

What risks am I concerned about?

Rather than just do a wholesale implementation of the old risk overlay, I thought it would be worthwhile considering from scratch what risks I am concerned about, and whether the original approach is the best way to avoid them. 

1. Expected risk being just too high. In the original risk overlay, I just calculated expected risk based on estimated correlations and standard deviations.
2. The risk that volatility will jump up. Specifically, the risk that it will jump to an extreme level. This risk was dealt with in the risk overlay by recalculating the expected risk, but replacing the current standard deviation with the '99' standard deviation: where % standard deviation would be for each instrument at the 99% percentile of it's distribution.
3. The risk that vol is too low compared to historic levels. I dealt with this previously in the 'endogenous' risk management eg within the system itself not the risk overlay, by not allowing a vol estimate to go below the 5% percentile of it's historic range.
4. A shock to correlations. For example, if we have positions on that look they are hedging, but correlations change to extreme values. This risk was dealt with in the risk overlay by recalculating the expected risk with a correlation matrix of +1,-1; whatever is worse.  

Here is what I decided to go with:

1. Expected risk too high: I will use the original risk overlay method
2. Vol jump risk: I will use the original method, the '99' standard deviation.
3. Vol too low risk: I have slready modified my vol estimate, as I noted above, so that it's a blend of historic and current vol. This removes the need for a percentile vol floor.
4. Correlation shock: I replaced with the equivalent calculation, a limit on sum(abs(annualised risk % each position)). This is quicker to calculate and more intuitive, but it's the same measure. Incidentally, capping the sum of absolute forecast values weighted by instrument weights would achieve a similar result, but I didn't want to use forecasts, to allow for this method being universal across different types of trading strategy.
5. Leverage risk (new): The risk of having positions whose notional exposure is an unreasonable amount of my capital. I deal with this by using specific position limits on individual instruments. 
6. Leverage risk 2 (new): The implication of a leverage limit is that there is a minimum % annualised risk we can have for a specific instrument. I deal with this by removing instruments whose risk is too low
7. Leverage risk 3: At an aggregate level in the risk overlay I also set a limit on sum(abs(notional exposure % each position)). Basically this is a crude leverage limit. So if I have $1,000,000 in notional positions, and $100,000 in capital, this would be 10.

Hence we have the following risk control measures:

• Endogenous
• Mixed vol estimate, including both long run and short run vol
• Risk overlay
• Normal: Estimated risk limit
• Jump: Estimated risk with 99% vol
• Correlation: Sum(abs(% annualised risk each position))
• Leverage: Sum(abs(notional exposure % each position))
• Off-line 
• Specific position limits to avoid any given instrument having a notional position that is too hight
• Instrument selection to avoid instruments whose % risk is too low

Off line 

Let's being by quickly talking about the 'off-line' risk controls: specific position limits, and minimum standard deviation.

Firstly, we only select instruments to trade that have some minimum standard deviation. 

It can be shown (see my forthcoming book for proof!) that there is a lower limit on the instrument risk (s), given a maximum resulting leverage ratio (size of notional position in that instrument divided by total capital) L, instrument weight w, annualised risk target (t), and instrument diversification multiplier (IDM). Assuming the ratio of maximum forecast to average forecast is 2:

Minimum s   =  (2 × IDM × w × t) ÷ L

For example, if the instrument weight is 1%, the IDM is 2.5, the risk target is 25%, and we don't want more than 100% of our notional in a specific instrument (L=1) then the minimum standard deviation is 2 * 2.5 * 0.01 * 0.25 / 1 = 1.25%. 

That's pretty low, but it would sweep up some short duration bonds. Here's the safest instruments in my data set right now:

EURIBOR         0.287279 *
SHATZ           0.553181 *
US2             0.791563 *
EDOLLAR         0.801186 *
JGB             1.143264 *
JGB-SGX-mini    1.242098 *
US3             1.423352
BTP3            1.622906
KR3             1.682838
BOBL            2.146153
CNH-onshore     2.490476

It looks like those marked with * ought to be excluded from trading. Incidentally, the Shatz and JGB mini are already excluded on the grounds of costs. 

This will affect the main backtest, since we'll eithier remove such instruments entirely, or - more likely - mark them as 'bad markets' which means I come up with optimal positions for them, but then don't actually allow the dynamic optimisation to utilise them.

Note that it's still possible for instruments to exceed the leverage limit, but this is dealt with by the off line hard position limits. Again, from my forthcoming book:

Maximum N =  L × Capital ÷ Notional exposure per contract 

          

Suppose for example we have capital of $100,000; and a notional exposure per contract of $10,000. We don’t want a leverage ratio above 1. This implies that our maximum position limit would be 10 contracts.

We can check this is consistent with the minimum percentage risk imposed in the example above with a 1% instrument weight, IDM of 2.5, and 25% risk target. The number of contracts we would hold if volatility was 1.25% would be (again, formula from my latest book although hopefully familar):

N_i = (Forecast ÷ 10) × Capital × IDM × weight_i × t ÷ 

       (Notional exposure per contract  × s_{%, i})

 = (20 ÷ 10) × 100000 × 2.5 × 0.01 × 0.25    ÷ (10,000 ×0.0125)  

 = 10 contracts

Because the limits depend on the notional size of each contract they ought to be updated regularly. I have code that calculates them automatically, and sets them in my database. The limits are used in the dynamic optimisation; plus I have other code that ensures they aren't exceeded by blocking trades which would otherwise cause positions to increase.

Note: These limits are not included in the main backtest code, and only affect production positions. It's possible in theory to test the effect of imposing these constraints in a backtest, but we'd need to allow these to be historically scaled. This isn't impossible, but it's a lot of code to write for little benefit.

Risk overlay

Once we've decided what to measure, we need to measure it, before we decide what limits to impose.

• Normal: Estimated risk limit
• Jump: Estimated risk with 99% vol
• Correlation: Sum(abs(% annualised risk each position))
• Leverage: Sum(abs(notional exposure % each position))

In my original post I noted that a way to set these limits without getting tempted to fit them, was to set them at some percentile of the distribution of each risk measure. I'm going to use the 99% percentile - the risk overlay is designed to kick in occasionally, not every other day.

I ran my full trading system (discussed here) to generate these plots; but as noted above the positions used for these calculations were unrounded positions before dynamic optimisation.

Let's start with 'normal' risk

Median: 16.9%

99% percentile: 34.9%

Notice we undershoot our risk target- 25% - and this is most likely because the IDM is capped at 2.5; so has more instruments are added we can't leverage up as much as is theoretically required.

It seems reasonable to cap this at 50% - twice the annual risk target - since we can take on positions that are twice as big than the average, if the forecast is large enough. And indeed that's what I did in the previous iteration of my risk overlay. However this would require all of our instruments to hit double forecast at the same time: unlikely! So let's stick with the 99% percentile approach and set this to 34.9/25 = 1.4 times the long run risk target. This wouldn't have been triggered in a backtest since the early 1980s.

It's quite interesting to also plot the realised risk:

Here the expected risk is in orange, and the realised risk over the last 6 months is in blue. You can see that they follow each other to an extent, but we clearly realise more risk especially in the period after 2005. The median of the realised risk is much closer to target: 27%. Anyway, that's nothing to do with this post....

Now for the 'jump' risk

Median: 36.8%

99% percentile: 89.5%

This equates to a risk limit of 89.5/25 = 3.6 times normal risk. It's a bit lower than the 6 times I used before, probably reflecting a more diversified portfolio. This would have been triggered in the late 1970's and early 1980's, plus in 2011, 2014 and 2021.

Now, correlation risk:

Median: 60.3%

99% percentile: 84.6%

This equates to a risk limit of 84.6/25 = 3.4 times normal risk. This would have been trigerred, briefly, in 2014.

Finally, leverage:

Remember the units here aren't % annualised risk, they are a ratio to capital.

Median 6.15

99% percentile: 13.9

This is a tricky one, and if you're going to use this approach I would advise checking for yourself since the leverage will depend very much on the mixture of instruments you have. The leverage has clearly grown over time as more instruments have been added with lower risk. On the face of it, 13.9 times leverage sounds pretty flipping scary. 

But remember that the use of position limits in production, and the avoidance of instruments with very low risk, will probably mean the real leverage isn't as bad as all this. For example, the leverage I have on at this very moment with my actual production positions after dynamic optimisation is just 1.6 times.

Anyway we have to use a number, so let's be a tiny bit more conservative than 99% percentile and go for lucky 13.

Here's the limits in the configuration in case you want to change them:

risk_overlay:
  max_risk_fraction_normal_risk: 1.4
  max_risk_fraction_stdev_risk: 3.6
  max_risk_limit_sum_abs_risk: 3.4
  max_risk_leverage: 13.0

Now, for each of the risk measures I calculate the following scalar: Min(1, Limit / Risk measure). 

That will be equal to 1 most of the time, and lower than 1 if the risk measure has exceeded the limit.

We'd then take the minimum value of these four ratios. Notice that different risk ratios are kicking in at different times, as different risk measures become problematic.

The longest period the risk overlay is turned on is for about 6 months in late 2014. Mostly though it just kicks in for a few days. The minimum value is 0.77; so we'd reduced our position by almost a quarter. The overlay is on about 4% of time; this feels about right given we have four risk measures kicking in 1% of the time, so that's what we'd expect if they were uncorrelated.

Finally I multiply the unrounded positions (pre dynamic optimisation) by this risk scaler value. 

There will be additional trading costs from implementing this; if I wasn't using the dynamic optimisation that provides a buffering service, I'd probably think about smoothing the risk overlay a little bit to reduce those costs.

I won't be bothering to test the effect of running this risk overlay; it doesn't turn on enough to give meaningful results. 

Summary

That's probably it in terms of modifying my flagship system for a while - probably a good thing after quite a few changes in a relatively short time!