Kelly versus Classical portfolio theory, and the two kinds of uncertainty premium

Since I was a young lad there has been an ongoing fight in Financial Academia 'n' Industry between two opposing camps:

• In the red corner are the Utilitarians. The people of classical finance, of efficient frontiers, of optimising for maximum return at some level of maximum risk.

• In the blue corner are the Kellyites. Worshipping at the feet of John Kelly and Ed Thorpe they have only one commandment in their holy book: Thou shalt maximise the expectation of log utility of wealth.

This post is sort of about that battle, but more generally it's about two different forms of uncertainty for which humans have varying degrees of stomach for, and how they should be accounted for when deciding how much volatility your trading or investment portfolio should have: "Risk" (which we can think of as known unknowns, or at least the amount of volatility expected from a risk & return model which is calibrated on past data) and "Uncertainty" (which we can think of as unknown unknowns, or to be more precise the unknowability of our risk & return model).

The Kellyites deny the existence of "risk appetite" (or at least they deny it's importance), whereas the Utilatarians embrace it. More seriously both camps seriously underestimate the importance of uncertainty; which will be more the focus of this post.

This might seem somewhat esoteric but in laymans term this post is about answering an extremely critical question, which can be phrased in several equivalent ways:

• What risk target should I have?
• How much leverage should I use?
• How much of my capital should I bet on a given position?

This post was inspired by a twitter thread on this subject and I am very grateful to Rob Hillman of Neuron Advisors for pointing me towards this. I've blogged about this battle before, here (where I essentially address one interesting criticism of Kelly) and I've also talked about Kelly generally here. 

If you're unfamiliar with arithmetic and geometric returns it's probably worth rereading the first part of this post here, otherwise you can ignore these other posts (for now!).

Classic Utilitarian portfolio optimisation

To make live easier I'm going to consider portfolios of a single asset. The main difference I want to highlight here is the level of leverage / risk that comes out of the two alternatives, rather than the composition of the portfolio.

I'm sure the readers of this blog don't need reminding of this but basically Utilitarians tend to do portfolio optimisation like this: specifying the investors utility function as return minus some penalty for variance. Which for a single asset with leverage, if we assume the risk free rate is zero or that the return is specified as an excess return (it doesn't matter for the purposes of this post) becomes this:

Maximise f.E(r) - b*[f.E(s)]^2

Where f is the leverage factor (f=1 incidentally means fully invested, f=2 means 100% leverage, and so on), r is the expected return, E is the expectation operator, risk s is measured as the standard deviation of returns on the unleveraged asset, b is the coefficient of risk aversion (you'll often see 1/2 here, but like, whatever). This is a quadratic utility function. In this model risk tolerance is an input, here defined as a coefficient of aversion.

Of course we could also use a different utility function, like one which cares about higher moments, but that will probably make the maths harder and definitely mean we have to somehow define further coefficients establishing an investors pain tolerance for skew and kurtosis.

This specification isn't so much in fashion in industry; it's hard enough getting your investors to tell you what their risk appetite is (few people intuitively understand what 150% standard deviation a year feels like, unless they're crypto investors or have money with Mr C. Odey). Imagine trying to get them to tell you what their coefficient of risk aversion is. Easier to say "our fund targets 15% a year volatility" which most people will at least pretend to understand. So we use this version instead:

Maximise portfolio returns = weights.E(asset returns)

Subject to: portfolio risk = function of weights and E(asset covariance)<= maximum risk

Which for one asset, with no risk free rate is:

Maximise f.E(r)

Subject to f.E(s)<=s_max

Where s_max is some exogenous maximum risk tolerance specified by the investor

Importantly (a) maximum risk depends on the individual investors utility function (which is assumed to be monotonically increasing in returns up to some maximum risk at which point it drops to zero - yeah I know, weird) and (b) the return here is arithmetic return (c) we only care about the first two moments since risk is measured using the standard deviation. Again risk is an input into this model (as a tolerance limit this time, rather than coefficient), and the optimal leverage comes out.

Kelly 

Under Kelly we choose to find the portfolio which maximises the expectation of the log of final wealth. For Gaussian returns (so again, not caring about the 3rd or higher moments which I won't do throughout this post) it can be shown that the optimal leverage factor f* is:

f* = (r - r_f) / s^2

(If you aren't in i.i.d. world then you can mess around with variations that account for higher moments, or just do what I do - bootstrap)

Where r is the expected arithmetic portfolio mean, r_f is the risk free rate and s is the standard deviation of portfolio returns without any leverage (and with expectations operators implicit - this is important!). f=1 incidentally means fully invested, f=2 means 100% leverage, and so on. Noting that the risk of a portfolio with leverage will be f*s this means we can solve for the target risk s*:

s* = f*. s = (r - r_f) / s

Notice that this thing on the right is now the Expected Sharpe Ratio. This is my favourite financial formula of all time: optimal Kelly risk target = Expected Sharpe Ratio. It has the purity of E=mc^2. But I digress. Let's take out the risk free rate for consistency:

f* = r  / s^2

s* = r  / s

Importantly for the battle in this world we don't specify any risk tolerance, or coefficient of risk aversion, or utility function. Assuming an investor wants to end up with the highest expected log utility of final wealth (or as I said here, the highest median expectation of final wealth) they should just use Kelly and be done with it.

The battle

Let's recap:

Kelly:       f* = r / s^2      s* = r / s
Utilitarian: f* = s_max / s    s* = s_max

With no risk free rate; f*= optimal leverage, s* = optimal risk, s_max is maximum risk (both standard deviations) and r = expected return.

Importantly the two formula don't usually give the same answers (unless r /s = s_max; i.e. the Sharpe Ratio is equal to the risk tolerance), and the relative answer depends on the Sharpe Ratio you're using versus typical risk appetite.

CASE ONE: Kelly leverage< Utilitarian leverage

If you're investing in a long only asset allocation portfolio then a conservative forward looking estimate of Sharpe Ratio (like those in my second book) would be about 0.20. If we assume expected return 2% and standard deviation 10% then the optimal Kelly risk target will be 20%, implying a leverage factor of 2. But that only gives 4% return! If the utilitarian investor is rather gung ho and has a risk appetite of 30% then the optimal leverage for them would be 3.

CASE TWO: Kelly leverage > Utilitarian leverage

If you're running a sophisticated quant fund with a lot of diversification and a relatively short holding period then a Sharpe Ratio of 1.0 may seem reasonable. For example if a stat-arb equity neutral portfolio has expected return 5% and standard deviation 5% (assuming risk free of zero) then the optimal Kelly risk target will be 100%, which implies a leverage factor of 20 (!). But a utilitarian investor may only have a risk appetite of 15%, in which case the optimal leverage will be 3. And indeed most equity neutral funds do run at leverage of about 3.

For case one I refer you back to my previous post, in which I said that nobody, no matter what their risk appetite should invest more than Kelly if you believe my logic that it is MEDIAN expected portfolio value that matters rather than mean. Essentially where the utility optimisation gives you a higher leverage than Kelly you should ignore it and go with Kelly.

Case two is a little more complicated, and the solution quoted in a thousand websites and papers is "Most investors find the risk of full Kelly to be too high - we recommend they use half Kelly instead". Frankly this is a bit of a cop-out by the Kelly people, which admits to the existence of risk appetite.

The compromise

I personally believe in risk appetite. I believe that people don't like lumpy returns, and some are more scared of them than others. Nobody has the self discipline to invest for 40 years and completely ignore their portfolio value changes in the interim.

But I also believe that using more than full Kelly is dangerous, insane, and wrong.

So this means the solution is easy. Your risk target should be:

s* = Minimum(r / s , s_max)

Where the first term is of course the Kelly optimum without the risk free, and the second is the risk tolerance beloved of utilitarian investors. And your leverage should be:

f* = Minimum(r / s^2, s_max / s)

What we know, and what we don't know

In case you haven't noticed I find this battle a little tiresome (hence my pretty superficial attempt at 'solving' it), and mainly because it completely ignores something incredibly important. We have two guys in the corner of a room arguing about whether Margin Call or The Big Short is the best film about the 2008 financial crisis (a pointless argument, because both v. good films), whilst a giant elephant is in the corner of the room. Running towards them, about to flatten them. Because they haven't seen it. They're too busy arguing. Have I made the point sufficiently, do you think?

What is the elephant in this particular metaphorical room. It's this. We don't know r. Or s. Of the Sharpe Ratio, r/s (ignoring risk free of course). And without knowing these figures, we don't have a hope in hell of finding the right leverage factor.

We have to come up with a model for them, based on historical data, because that's what quant finance people do. Which means there is the risk that:

• It's the wrong model (non Gaussian returns, jumps, autocorrelation...)
• The parameters of the model aren't stable
• The parameters of the model can't be accurately measured

This triumvirate of problems should be recognisable to people familiar with my work, and you already know that I feel it is most productive to focus on the third problem for which we have relatively straightforward ways of quantifying our difficulties (using the classical statistical workhorse of sampling distribution).

Parameter uncertainty (and the other issues) isn't such an issue for standard deviation; we are relatively good in finance at predicting risk using past data (R^2 of regressions of monthly standard deviation on the previous month is around 0.6, compared to about 0.01 for means and Sharpe Ratios). So let's pretend that we know the standard deviation.

However the Sharpe Ratio is the key factor in working out the optimal leverage and risk target for Kelly (which even for Utilitarians should act as a ceiling on your aspirations). The sampling distribution of Sharpe Ratio is highly uncertain.

The effect of parameter uncertainty on Sharpe Ratio estimates

There is an easy closed form formula for the variance of the Sharpe Ratio estimate under i.i.d returns given N returns:

w = (1+ 0.5SR^2)/N

(If you aren't in i.i.d. world then you can mess around with formulas that account for higher moments, or just do what I do - bootstrap)

We need an example. Let's just pick an annual Sharpe Ratio out of the air: 0.5. And assume the standard deviation is 10%. And 10 years of monthly data. But if you don't like these figures feel free to play with the example here in google docs land (don't ask for edit access - make your own copy).

Here is the distribution of our Sharpe Ratio estimate:

The concept of "Uncertainty appetite"

Now let's take the distribution of Sharpe Ratio estimate, and map it to the appropriate Kelly risk target:

Yeah, of course it's the same plot, since s* = r/s = SR. You should however mentally block off the negative part of the x-axis, since we wouldn't bother running the strategy here and negative standard deviation is meaningless. And here is the plot for optimal leverage (r/s^2):

So to summarise the mean (and median, as these things are Gaussian regardless of the underlying return series) optimal risk target is 50% and the mean optimal leverage is 5. Negative leverage sort of makes sense in this plot, since if an asset was expected to lose money we'd short it.

At this point the Kellyites would say "So use leverage of 5 or if you're some kind of wuss use half Kelly which is 2.5" and the Utilitarians might sniff and say "But my maximum risk appetite is 15% so I'm going to use leverage of 1.5". Since the optimal standard deviation of 50% is relatively high it's very likely that we'd get a conflict between the two approaches here.

But we're going to go beyond that, and note that there is actually a lot of uncertainty about what the optimal leverage and risk target should be. To address this let's introduce the concept of uncertainty appetite. This is how comfortable investors are with not knowing exactly what their optimal leverage should be. It is analogous to the more well known risk appetite, which is how comfortable investors are with lumpy returns.

Someone who is uncertainty blind would happily use the median points from the above distributions- they'd use full Kelly, assuming of course that their risk appetite wasn't constraining them to a lower figure. And someone weird who is uncertainty loving might gamble and assume that the true SR lies somewhere to the right of the median, and use a higher leverage and risk target than full Kelly.

But most people will have a coefficient of uncertainty aversion (see what I did there?). They'll be uncomfortable with full Kelly, knowing that there is a 50% chance that they will actually be over gearing. We have to specify a confidence interval that we'd use to derive the optimal leverage, with uncertainty aversion factored in.

So for example suppose you want to be 75% sure that you're not over-geared. Then you'd take the 25th percentile point off the above distributions: which gives you an expected Sharpe Ratio of about 0.29, a risk target of 29% and optimal leverage of 2.9.

Here are a few more figures for varying degrees of uncertainty:

Confidence interval         Optimal risk       Optimal leverage 

       <5.8%                     Don't invest anything
       10.0%                   9.2%                 0.93
       15.0%                  17.1%                 1.71
       20.0%                  23.2%                 2.32
       30.0%                  33.3%                 3.33

       40.0%                  41.9%                 4.19

       50.0%                  50.0%                 5.00 

Incidentally the famous half Kelly (a leverage of 2.5) corresponds to a confidence interval of about 22%. However this isn't a universal truth, and the result will be different for other Sharpe Ratios.

What this means in practice is that if you're particularly averse to uncertainty then you'll end up with a pretty low optimal Kelly risk target. How does this now interact with risk appetite, and the Utilitarian idea of maximum risk tolerance? Well the higher someones aversion to uncertainty, the lower their optimal risk target will be, and the less likely that an exogenous maximum risk appetite will come into play.

Now someone who is averse to uncertainty will probably also be averse to the classical risk of lumpy returns. You can imagine people who are uncertainty averse but not risk averse (indeed I am such a person), and others who are risk averse but not uncertainty averse, but generally the two probably go together. Which also raises an interesting philosophical point about the difference between them, as we'd rarely be able to distinguish between the two kinds of uncertainty except in specific experiments or unusual corner cases.

Summary

Both the Kellyites and the Utilitarians have good points to make - you should never bet more than full Kelly no matter how gung ho you are, and risk appetite is actually a thing even if few investors really know have quadratic utility functions.

But both are missing the real point, which is that there is a lot of uncertainty about what the Sharpe Ratio and hence optimal leverage really is. Assuming some conservatism and a degree of uncertainty appetite this produces a Kelly optimal revised for uncertainty which will be lower than the uncertainty blind full Kelly. This then makes risk appetite less relevant as a constraint, and the whole battle becomes a moot point.