This might seem boring, but answering this will help us with some other serious problems: Does diversification increase the expected value of your portfolio or just reduce the volatility? If so can we then afford to pay extra costs to get diversification? Does adding a small amount of bonds to an all equities portfolio increase your likely returns?

It turns out that the answer to this boils down to one of the most fundamental questions in financial economics:

**How should we evaluate the expected value of possible outcomes?**

## A brief introduction to geometric returns

When thinking about past and future returns I'm going to be using geometric means rather than the more common arithmetic means. Geometric means reflect what you will actually earn over time.

To understand this better let's look at an example. Consider an investment in which you invest $100 and earn 30%, 30% and -30% over the next three years of returns. The

**arithmetic mean of returns**is the sum of annual returns, 30% + 30% - 30% = 30%, divided by the number of years (3), which

**equals 10%**. You might expect to have an extra $30 after three years: probably more with the magic of compound interest.

End of year 1: $100 + 30% * $100 = $130

End of year 2: $130 + 30% * $130 = $169

End of year 3: $169 - 30% * $169 = $118.30

Whoops. Compound interest is a wonderful thing but it magnifies losses as well as gains. Now suppose you'd made a pathetic 5.76% a year but consistently:

End of year 1: $100 + 5.76% * $100 = $105.76

End of year 2: $105.76 + 5.76% * $105.76 = $111.85

End of year 3: $111.85 + 5.76% * $111.85 = $118.30

**consistent return that gives the correct final account value**. So the

**geometric mean**of 30%, 30% and -30% is

**5.76% a year**.

### The bit of the post where I put an obligatory equation or two

Mathematically the geometric mean is [n√(1+ r1)(1+ r2)....(1+ rT)]-1 where rt are each of T returns. Alternatively it's exp[(1 ÷ N) Σln(1+rt))] – 1 where ln is the natural log, and exp is the exponent function.

Notice that the geometric mean is a concave function of the final value of the portfolio (1+ r1)(1+ r2)....(1+ rT). This is an important point which I'll return to later.

### Some irony

I am unlikely cheerleader for geometric returns... until a couple of years ago I'd never actually used them! That's because in the hedge fund world where we rebalance to target expected risk on constant capital it's better to use non compounded curves (See this post for more).

It's not necessary to use geometric returns since there no compounding and the volatility of different options is identical (expected risk on target), rather you can use arithmetic returns and make your life easier (you can also focus entirely on Sharpe Ratios, since you effectively have as much leverage as you need to maximise returns for a given level of risk target).

## Some interesting properties of geometric returns

Geometric returns give a more realistic picture than mean returns. To take an extreme example consider the following series of returns: 100%, 100%, -100%. The arithmetic return is 100%: What a fantastic investment! But the geometric mean is easy to calculate: 0%. You will have nothing left after three years have passed.

Geometric means are always lower than the arithmetic mean, unless all annual returns are identical. The difference between the two measures is larger for more volatile assets. In fact we can see this easily with the following which is a nice approximation for geometric means:

μg = μa – 0.5 σ_2

...where μg is the geometric mean, μa is the arithmetic mean and σ_2 is the variance of returns. In other words the geometric mean is the arithmetic mean, less a correction for risk.

(This can be proven using Jensens Inequality which I'll return to later)

**the benefits of diversification are greater when average returns are measured with geometric means.**

## The consequences of using geometric returns

### 1) Diversification improves returns: so we can afford to pay for it

Take a group of similar assets, like equities in the same country and sector. It's not unreasonable to assume they have equal arithmetic returns, equal standard deviations (and thus equal Sharpe Ratios - and equal geometric means) and identical correlations. The optimal portfolio here has equal weights

**and as many assets as possible**. But adding these assets doesn't affect the arithmetic mean, which is unchanged. It reduces Sharpe Ratio, rapidly. But is also improves the geometric mean, a little more gradually.

For example, assuming correlation of 0.85:

1 asset: arithmetic mean 5%, geometric mean 1.3%

5 assets: arithmetic mean 5%, geometric mean 1.8%

We can also use geometric returns to "pay" for higher diversification costs. If you can get an extra 0.5% in geometric returns then you can pay 0.4% more in costs and still be ahead.

1 asset: arithmetic mean 5%, geometric mean 1.3%

5 assets: arithmetic mean 5% - 0.4% = 4.6%, geometric mean 1.4%

### 2) 100% equity portfolios are bad even if you don't maximise Sharpe Ratio

Also using geometric returns is part of the way past the classic portfolio optimisation quandry: should we opt for a portfolio with higher return (more equities), or lower risk (more bonds)? The maximum Sharpe Ratio portfolio is just one possible compromise between these two options. But for those with a higher tolerance for risk it is inferior to options with more return.

The maximum geometric mean portfolio is interesting. It's the portfolio for which there is no point increasing risk further, even if doing so gives you a higher arithmetic return.

So an interesting implication is that adding a small amount of bonds to an all equities portfolio will increase geometric return: or to put it another way all investors should own some bonds.

For example given the following properties:

Bonds: arith. mean 1.6%, standard deviation 8.3%, geo. mean 1.3% Geometric Sharpe Ratio 0.15

Equities: arith. mean 5%, standard deviation 19.8%, geo. mean 3% Geometric Sharpe Ratio 0.15

(these are derived from 100+ years of US real returns, adjusted to reflect more realistic forward expectations and to equalise geometric Sharpe Ratio)

A portfolio with 20% in bonds will have the following properties:

80:20 portfolio: standard deviation 16%, geo. mean 3% Geometric Sharpe Ratio 0.188

Geometric mean (y axis) as bonds added to an all equity portfolio (x axis: 0= no bonds, 1.0 = 100% bonds) |

## The bear case for using geometric returns

The above findings are, relatively startling. So it's important that geometric return is in fact "real". But there is some debate about this. I was prompted to write this post after being shown this paper:

http://www.bfjlaward.com/pdf/25968/65-76_Chambers_JPM_0719.pdf

...h/t to Daal on elitetrader.com.

It's a very involved paper which also covers "rebalancing return" but here are the key points in relation to diversification:

High AM: Arith. mean 5%, standard deviation 15%

Low AM: Arith. mean 4.375%, standard deviation 10%

(The distribution of expected arithmetic means is Gaussian with lower standard deviation for more volatile assets)

(Again the distribution of expected geometric means is Gaussian with lower standard deviation for more volatile assets)

You can see that both assets have the same expected geometric mean, though there is more uncertainty about the estimate for the higher volatility "High AM" asset.

Finally let's have a look at the distribution of final portfolio values:

(A final value of 1.0 indicates the portfolio hasn't grown, 2 means 100% growth over 10 years and so on)

Now this distribution is more interesting. Even if you squint really hard it isn't Gaussian - it's a skewed lognormal. The bunching of values on the left hand side is happening when a lot of losses occur in a row. Because we're not using leverage the portfolio value can't go below zero; hence we get bunching.

The means of the distribution are: 1.65 (High AM) and 1.55 (Low AM). These are the key numbers. Although both assets have the same geometric mean

Here is clear evidence that the

It's a very involved paper which also covers "rebalancing return" but here are the key points in relation to diversification:

- A key misconception concerning the expected geometric mean return is that it provides an accurate indication of long-term expected future wealth.
- Another potential misconception regarding geometric mean returns is that maximization of a portfolio’s expected geometric mean return is an optimal portfolio strategy
- An asset’s expected geometric mean return (i.e., the expected compounded rate of return) is the probability-weighted average of all of the potential realized geometric mean returns.
- ...volatility does not diminish expected value.

- geometric mean scales (concavely) with final portfolio value.
- BUT
**expected**geometric mean does not scale with**expected**portfolio value - Therefore
**maximising expected geometric mean might not maximise expected portfolio value** - Instead
**maximising expected arithmetic mean will maximise portfolio value**

High AM: Arith. mean 5%, standard deviation 15%

Low AM: Arith. mean 4.375%, standard deviation 10%

These apparently arbitrary values have been chosen so that

**both assets have the same geometric mean**.
Now I'm going to plot the distribution of statistical estimates from this little Monte Carlo exercise (with 500,000 runs; to ensure reasonably smooth results)

First the distribution of arithmetic means:

(The distribution of expected arithmetic means is Gaussian with lower standard deviation for more volatile assets)

The mean of the distribution (i.e. the expected arithmetic mean returns) are: 4.99% (High AM) and 4.38% (Low AM). This verifies that the Monte Carlo hasn't done something weird.

Now geometric means

(Again the distribution of expected geometric means is Gaussian with lower standard deviation for more volatile assets)

Finally let's have a look at the distribution of final portfolio values:

Now this distribution is more interesting. Even if you squint really hard it isn't Gaussian - it's a skewed lognormal. The bunching of values on the left hand side is happening when a lot of losses occur in a row. Because we're not using leverage the portfolio value can't go below zero; hence we get bunching.

The means of the distribution are: 1.65 (High AM) and 1.55 (Low AM). These are the key numbers. Although both assets have the same geometric mean

**the final portfolio value is larger for the asset with a higher arithmetic mean.****highest expected final value comes when the arithmetic mean is higher even with higher volatility**. From the paper:- Another potential misconception regarding geometric mean returns is that maximization of a portfolio’s expected geometric mean return is an optimal portfolio strategy

## The case for the defence - a question of measuring expectations

I'm going to home in on one particular implication of the paper cited above:

- The
**expected**final value of the portfolio**is the probability-weighted average**of all of the possible portfolio final values.

Expectation: A word we use a lot in economics and finance without a pause. What does it mean? And also, and very importantly, which average?

When I was at school we learned about three: the mean, the median, and the mode (which I won't be using here). Remember from the figure above that the distribution of realised final values is right skewed. Hence the mean will be greater than the median. So the choice of average matters a lot.

When I was at school we learned about three: the mean, the median, and the mode (which I won't be using here). Remember from the figure above that the distribution of realised final values is right skewed. Hence the mean will be greater than the median. So the choice of average matters a lot.

The paper assumes that the probability weighted average of all potential portfolio values is the

**.**

__mean__of the distribution of possible portfolio valuesTo be clear then:

- The expected final value of the portfolio
**is the probability-weighted average**of all of the potential realized portfolio final values. In the paper this is the**mean of the distribution of possible portfolio values**.

## The case for using the median not the mean

Is the mean really appropriate? Personally I would say no, for two reasons:

- Risk neutral behaviour doesn't really exist
- The median is closer to how humans form expectations

### Does risk neutral behaviour exist?

Using the mean makes sense for

**risk neutral**investors. Let's take a simple and rather extreme example. Suppose your entire wealth is £100,000. I offer you the chance to buy a lottery ticket for £100,000, which will pay you £100 million, at odds of 999 to one. The expected value of the ticket

**using the weighted average**is £100,100. For an economist this bet is worth taking!

__mean__of the outcomesThe two options are:

- Don't buy the ticket. Mean of future wealth: £100,000. Median of future wealth: £100,000
- Buy the ticket. Mean of future wealth: £100,100. Median of future wealth: £0

- Risk averse who require paying more than the risk neutral mean of outcomes to take on more risk (they would want to buy the lottery ticket for less than £100,000)
- Risk neutral investors who use the weighted average mean to evaluate options
- Risk lovers who are happy to pay to take on more risk (they would happily pay more than £100,000 for the ticket)

**are risk lovers then there must also be risk neutral investors**; it makes no sense to have a continuum with a break in the middle.

Looking around it does look like some people

*seem*to love risk to the point they'll happily pay for it: eg gambling in casinos when the odds are against them (which they nearly always are). If these weirdos exist it does seem more plausible that risk neutral investors also exist. However I would argue that

**true risk loving behaviour**doesn't exist.

Instead this behaviour is a result of people

**misjudging probabilities**due to

**cognitive biases**in the way we think about risk. The cognitive science hadn't been incorporated into financial economics when the idea of the continuum above was proposed.

We know that people overestimate the likelihood of events with very small probabilities (which is one reason why people do buy lottery tickets which cost only a fraction of their expected value even though they always have a negative expected value, and buy insurance against terrorist attacks).

If you ask a desperate gambler who is about to put the last of their money into the slot machine if they expect to win on this spin their answer will be "of course"; probably because they suffer from gamblers fallacy and believe they are "due" a win.

Similarly the most aggressive investors invest in highly speculative portfolios of rubbish companies with almost no diversification; which on the face of it would only make sense if they were risk loving.

But I would argue - again - that is a failure of probability assessment. Yes the investors say we know that diversification is better, but we have skill and can pick the best stocks. They

**overestimate their probability**of beating the market - all of them are above average - the Lake Wobegon effect.Cognitive failure leading to probability mis-assessment is mistaken for risk loving behaviour by economists. This is wrong.

**With no risk loving investors**

**I also believe risk neutral behaviour is also a myth.**In reality

**everyone requires some compensation for risk**.

### The median makes more sense to humans than the mean

As humans when we think about expectations it is the median that we are thinking about. If the weather forecast tomorrow is for a 10% chance of rain, and I ask someone

**what they expect the weather to be**, they will say they expect it to be dry (the median outcome). They won't say they expect it to be a little bit wet (the mean outcome).

The £100K lottery may be an extreme example but as we've already seen future wealth is always fairly heavily right skewed; enough so that the difference between mean and median is pretty significant.

Cross sectional distribution of wealth and income in the real world is also famously right skewed. Would you want to live in a country where there is a tiny chance of being very wealthy, but you're most likely to be dirt poor? (Hint: net migration from the very equal Nordic states to much less equal America is almost zero).

Would you want to do a job where you have a miniscule chance of earning millions, but will probably barely earn a living wage? (Again a lot of kids - or their parents - want to be professional footballers, but this is a judgement error that comes from overestimating the probability that they personally will make it to the top leagues).

I think the correct way to evaluate future wealth outcomes is by using the

**median**. To most people the idea of "

**I expect what will happen is what is likely to happen half the time**" is a more natural concept of expectations than "probability weighted average mean".

In an ideal world you'd show people distributions and explain the uncertainty involved and then get an idea of their risk / reward payoff function. But short of that I'd say that

**even the least risk averse humans on the planet should use the median outcome when evaluating future investments.**## Implications for using the median rather than the mean

Returning to the plots above what figures do we get if we summarise the for the median rather than the mean? Remember that High AM has a higher arithmetic mean than Low AM, but both have the same geometric mean.

Arithmetic mean:

High AM: Mean 4.99 Median 5.0

Low AM: Mean 4.38 Median 4.38

Geometric mean:

High AM: Mean 4.06 Median 3.95

Low AM: Mean 3.95 Median 3.96

High AM: Mean 1.647 Median

**1.473**
Low AM: Mean 1.549 Median

**1.474****The expected value (using the median) of future wealth is identical when geometric returns are identical, even if the arithmetic mean is lower.**

So

**maximising geometric mean will also maximise final portfolio value**. In other words the implications of using geometric mean that I outlined above still hold:

- We can use diversification to pay for higher costs
- 100% equity portfolios are not as good as portfolios with some bonds

## Summary

I think declaring the death of geometric returns is somewhat premature. It's true that using the classical economists view of expectation - the mean of the distribution of portfolio values - implies that final value isn't lowered by volatility. But this vanishes when you use the median of the distribution as your basis for expectation.

I feel personally that using the median, rather than the mean, is the correct approach. However this is an ideological debate - there is no right answer. Ultimately an economic model is a simplification of the vastly complicated reality of human behaviour.

quote from article: "Hence the mean will be greater than the mean."

ReplyDeleteAre you sure about this? Or is this a typo?

Typo. Fixed! Thanks!

DeleteFantastic article. I've long felt that, for a single person, the median outcome is all that matters. After all, the person has a sample of 1 in the cross section. For example, a single person should plan to their median remaining lifespan, but an insurance company should plan to the mean expected lifespan. From this perspective, a pension company might optimize on mean wealth, because it represents a large cross section of Investors. But a single investor should optimize on the median outcome. Haven't quite figured out how to prove it yet. Should probably talk to an actuary. Thanks again for the article.

ReplyDeleteHi Rob,

ReplyDeleteim having a problem with get_IB_market_data function.

When I try to capture bid-ask prices for ZS contract (ZS MAR 17), sometimes it gives me the following output:

2017-02-09 17:02:30.460000

Bid size, Ask size; Bid price; Ask price

[12, 27, 1050.0, 1050.0]

Since the market is not close, the correct would be something like this:

2017-02-09 17:02:30.460000

Bid size, Ask size; Bid price; Ask price

[12, 27, 1051.50, 1051.75]

Have you ever had this kind of problem?

The problem being the bid and ask price are the same? Sure its possible to happen for a fraction of a second until people update their prices.

DeleteI'm having this problem only for ZS. Can you explain me why does it happen? Im trying to run a loop the same way you did here: http://qoppac.blogspot.com.br/2014/10/the-worlds-simplest-execution-algorithim.html

DeleteAn example might be that the order book is at 1051.5 - 1051.75 with 10 lots on each side. Someone hits the bid at 1051.5 with a 15 lot order. The first 10 lots are filled and the 1051.5 quote is no longer the best bid. The remaining 5 lots offered at 1051.5 become the best offer.

DeleteThe true market is now an offer at 1051.5 for 5 lots with nothing on the bid. But the price capturing algo still has a memory of the bid at 1051.5. So it displays 1051.5 - 1051.5

The last bid quote doesn't get 'deleted'; it can only be superseded once a new best bid appears in the market. If there wasn't a bid already in the order book at say 1051.25 then there is no new best bid. Sometimes the IB API will then send a 'NA' bid price but sometimes it takes a while for this to happen.

It's more likely in thinly traded markets without much depth since quotes can remain stale for longer.

The way to deal with this is to remove prices which are stale, say more than a few seconds old, and replace them yourself with NAs.

Even though your answer makes perfect sense, I guess it's not what I'm getting here this time. I think there is something wrong with the function output because when inside of a loop of get_IB_market_data, it seems that sometime the function freezes and shows a standardized bid-ask price. Below is the loop output from this function.

Deletedef getprices(contract,tradesize):

sideprice=float('nan')

offsideprice=float('nan')

while math.isnan(sideprice) or math.isnan(offsideprice):

ans1=client.get_IB_market_data(ibcontract)

if tradesize>0:

sideprice=ans1[3]

offsideprice=ans1[2]

elif tradesize<0:

sideprice=ans1[2]

offsideprice=ans1[3]

return sideprice,offsideprice

...

sideprice : offsideprice

1062.0 1061.75

Ordem LMT: 1061.75

sideprice : offsideprice

1062.0 1061.75

Ordem LMT: 1061.75

sideprice : offsideprice

1050.0 1061.75

Ordem LMT: 1061.75

sideprice : offsideprice

1062.0 1061.75

Ordem LMT: 1061.75

sideprice : offsideprice

1062.0 1061.75

Ordem LMT: 1061.75

...

If you want I can email you with more details.

Martin

I just found a small bug in the code (wrapperv3.py). Try pulling from github and seeing if the issue goes away.

DeleteNow when the 1050 quote appears, it appears always to both the bid and ask at the same time.

DeleteI have changed get_IB_market_data including the line isinstance(self.cb.data_tickdata[tickerid][0], int) here:

while not finished and not iserror:

iserror=self.cb.flag_iserror

if isinstance(self.cb.data_tickdata[tickerid][0], int) or (time.time() - start_time) > seconds:

finished=True

pass

self.tws.cancelMktData(tickerid)

Its similar to your previous version of this function. Is this the reason why this problem is happening?

Sorry I don't recognise the code you are showing... is it from https://github.com/robcarver17/ibswigsystematicexamples/blob/master/sysIB/wrapper_v3.py ? If not I can't help you.

DeleteIts a modified version of your code. From github your code is:

Deletedef get_IB_market_data(self, ibcontract, seconds=30, tickerid=MEANINGLESS_ID):

...

while not finished and not iserror:

iserror=self.cb.flag_iserror

if (time.time() - start_time) > seconds:

finished=True

pass

self.tws.cancelMktData(tickerid)

marketdata=self.cb.data_tickdata[tickerid]

...

But I changed it to be similar to an old version where the conditional if didnt just waited an amount of seconds but also looked if any value was captured by data_tickdata[tickerid][0].

The error is not coming from my modification, I just tested it. So, I have no idea where this problem is coming from, maybe its an issue with IB datafeed.

DeleteHmmm. Its very hard to debug this kind of code. I'd suggest changing it so you don't terminate the loop until you have a bid and an ask at the same time, and you also wipe stale quotes out to ensure they're real and syncrhonised.

DeleteTo be honest: If you're going to actually use this to create an execution algo then you're probably going to need to do something a lot more complicated than this. Specifically you will need proper event driven code rather than pseudo event driven.

At some point in the future there will be code to do this on this blog...

Yes, I understand. I'm just trying to get a snapshot from the market with this code, I did it succesfully for all other assets but I'm getting this error for ZS. Anyway, thanks for your help.

DeleteI do occasionally get weird stuff from the data feed i.e. a price of 0. But that is easy to filter out. The general chatter on the web is that the data feed isn't as reliable as others, and if you're doing proper low latency trading you certainly can't use it.

DeleteHowever for my purposes it's worked well enough.

I'd like to thank you also as we did together find this bug!!!

Great post thank you!

ReplyDeleteWhat are your thoughts on using safe withdrawal rate as a benchmark for different investment options? To me this seems most relevant to people living off of their investments.

One interesting thing I noticed is that safe withdrawal rate didn't seem to vary between say an 80/20 equity/bond split and 100% equities using historical sequence of returns in analysis tools like cFireSim (http://www.cfiresim.com/).

What are your thoughts about this? Given how poor bond yields are currently I'm in a quandary about this.

If you take the view that you can't predict risk adjusted returns then if anything someone withdrawing money should go for a safer portfolio (80/20 rather than 100/0).

DeleteIts dangerous to start having subjective "poor bond yields" type thoughts. You should eithier use a systematic model to forecast returns or not bother.

Actually at an asset class level relative yield is a very poor measure of performance compared to say 12 month momentum. Momentum is still positive for bonds, albiet not as strong as in equities.

All this might justify a reduction from a strategic 80:20 split to a tactical 85:15 split, but no bigger change than that.

... or you could just wait for the next book which will discuss this in much more detail.

Hi Rob,

ReplyDeleteShouldn’t the Sharpe Ratio use the geometric mean since it calculates off of percent averages?

I use the geometric mean to measure my accounts avg. daily account growth since it is based off of daily percent averages. For instance, 4% growth represented by 1.04 and a 3% drawdown represented by 0.97, etc... Is this wrong? Should I be using the arithmetic mean?

Please advise. Thank you.

No you should use the geometric mean in the Sharpe Ratio, what you are doing is correct.

ReplyDeleteThank you Rob.

ReplyDeleteSorry for the belated response, I thought I signed up for notifications but I guess I didn't.

Anyway, I never hear of people using the geometric mean w/ the Sharpe Ratio (perhaps due to my ignorance), which is why I asked the question.

Thanks again.