Does X work, some brief thoughts and choose your adventure

When I was a spotty teenager I was a walking nerd cliche. I liked computers; both for programming and games. I was terrified of girls. I was rubbish at nearly all sports*.  And I played D&D (and Tunnels and Trolls, and Runequest).

* Nearly all: Not, I'm not talking about the 'sport' of Chess: I was also rubbish at Chess and still am. But due to some weird anomaly I was a dinghy sailing champion at school, and later world champion.

I also remember reading the Fighting Fantasy books written by Games Workshop founders Steve Jackson and Ian Livingstone. 

Copyright image used without permission, but if you click here you can buy this book so that seems fair

These books were 'choose your adventure' style. So you'd read page 1 and it would say something like 'You are outside a mysterious castle... long dull description of castle follows.... Do you (a) climb the castle wall (p.34), (b) dress up as a washerwoman and try and enter the castle through the front gate (p.172) or (c) attack the sentries head on' (p.91). And after making your choice you'd turn to page 34, or 172, or 91; and you'd find out what the consequences of your actions were, and there would be a new set of options unless you'd already died (hint: option (c) is a poor choice).

As well as playing the book properly you could do 'fun' stuff like reverse engineering the network graph for the pages and working out the optimal shortest route through the book.

Now anyone under the age of 35 will be open mouthed at how archaic this sounds, but yes, this is what we had to do for entertainment in the 1980's. And that was partly because even then parents were worred about screen time, and if you were curled up in a corner with an actual book they would be happier than it you were sat in front of your ZX spectrum playing Jet Set Willy (kids: google it). Of course now I spend entire days in front of a screen, and my idea of relaxation is to read books, so go figure.

You may be wondering what this has to do with anything, but bear with me for a bit as I'm now going to radically change the subject. 

Quite a few of the queries I get asked run along the lines of 'Does X make sense?'. Here are a few recent examples:

• Does changing your moving average speed make sense in different periods of volatility?
• Does it make sense to use a different system on the long or the short side?
• Does it make sense to use a different system in bull or bear markets?
• Should I use different parameters for different instruments?

These 'does it make sense' questions, are interesting. For starters, they all seem to make intuitive sense. It seems crazy that the market would behave in exactly the same way in periods of high and low vol. It's obvious the market doesn't go up and down in the same way. And naturally, trading Eurodollar is completely different from trading VIX. 

These 'does it make sense' questions are also dangerous. Every single one of them is an invitation to make your system more complex and less intuitive. Every single one is replete with the opportunity to overfit. They introduce a new set of parameters, to define market state; and then multiply the existing set of parameters by allowing different values for different states.

These questions are also complicated, and involve answer several different questions. They involve modifying or changing your system according to some kind of exogenous input, but exactly what that input should be isn't always obvious. It isn't always clear how we should make the change. And they also bring up some interesting questions about how we should evaluate the relevant changes.

Take the first question as an example. First of all we need to define what a different level of volatility looks like, how it should be measured, how many states there should be, and so on. There is room for a lot of implicit fitting there, so we should probably keep things simple and try and stick to some predefined measure; maybe 2 to 4 states of volatility using quartile measures based on the full history of the relevant instrument.

We then need to work out how to change our moving average speed (which for once is a very tightly defined 'how'). For me that at least is relatively straightforward: I'd change the forecast weights that determine how my forecasts are linearly combined. That at least can be done with explicit fitting to avoid exploding the number of parameters we have to consider. 

As for evaluation, well that isn't so bad, we can just compare a system before and after. Of course we need to decide if we're just going to look at Sharpe Ratio, or do we also care about skew. Also, are we happy with any improvement no matter how small? Or should we seek a higher threshold given we're making our system more complicated? Or is this a change that makes sense in it's own right, and we should be happy to do it even if it performs slightly worse (although not worse in a statistically significant sense).

What about the long and short system? Defining long and short is easy enough, but what parameters should we change? Something like the response function to the forecast (this kind of plot) perhaps. That might mean that we take on smaller positions when short in certain markets, which I suppose is what people would like and expect to see. 

However, should we evaluate such a change based on pure Sharpe Ratio? If we did this, then we'd end up never going short markets which have barely gone down in the past, and basically increase our long bias to eg bonds (we can make things worse, incidentally, by not forcing the response function to go through zero). That would only make sense for someone who is only investing in this trading strategy, and doesn't also have something like a 60:40 portfolio already. 

So a better evaluation would be something like 'alpha', where we are looking for improved returns versus the market rather than outright improvements (of course how we define 'the market' is another moot point).

Bull and bear markets is a related question. The first question, once again, is how to define bull and bear in a non forward looking way. Something like the risk adjusted 200 day EWMA seems reasonable without being tempted down the path of overfitting. I used changes in US interest rates in this post as a proxy (something that's relevant after the wild changes in rates products over the last few days.

How should we implement the change? I'd be most tempted to allow my forecast weights to change; rather than modifying the behaviour of any individual rule. And again, evaluation should rightly consider 'alpha' not just outright Sharpe Ratio. And consistency of performance should probably come into it; higher SR isn't as good as seeing better performance in bear markets as that 'insurance' payoff is part of what people like to buy CTA type strategies for.

Different parameters for different markets is another massive can of worms. The same issues of how to change things and how to evaluate them is relevant. Again, I'd be most tempted to fit different forecast weights, which I do a bit anyway because different instruments have different cost levels and I take those into account. 

But there is surely some value in pooling data from other markets as well? Shouldn't we fit based on some blend of an instruments own data, plus data from other markets (where the blend would be different depending on how much data the relevant instrument as). Should we also give higher weight to markets that are more similar? (same asset class, same country, in some kind of correlation cluster....?)

Also, what if we do this and one or more of the other changes? Should we trade long/short markets differently for US 10 year bonds as we do for VIX? 

Phew!

As you can see, these 'does X work' questions can quickly become very complicated. So you can see why I'm wary of doing them.

But I've decided to one of these investigations this month, and you get to choose which one!

Yes, just for this month, I'm going to make this a 'choose your adventure' blog where you get to decide the ending. These are the options:

• Does changing your moving average speed make sense in different periods of volatility?
• Does it make sense to use a different system on the long or the short side?
• Does it make sense to use a different system in bull or bear markets?

I haven't included 'fitting for instruments' since that is a significant project rather than something I can do in a few days, although I will probably look at it in the future.

Let me know which of the ideas above you'd like me to investigate, and I'll write a longer blog post about the one that is the most popular. The poll is here on twitter and on quiz-maker....

UPDATE 3rd March 11:47 GMT: VOTING CLOSED

The votes are in! The votes are (in traditional reverse order):

• Does it make sense to use a different system in bull or bear markets? 23.8%
• .... to use a different system on the long or the short side? 32.9% 
• ... to change your moving average speed make sense in different periods of volatility? 43.3 votes is the winner

 The vote breakdown is here, including Twitter (TWTR), quiz-maker (QM) and below the line (BTL) comments:

        TWTR BTL QM Total

Vol & speed 44 18 62

Long & Short 33 3 11 47

Bull & Bear 28 6 34

So at some point in the very near future I'll be posting on "Does changing your moving average speed make sense in different periods of volatility?"

UPDATE 4th March

Here's the post: https://qoppac.blogspot.com/2021/03/does-it-make-sense-to-change-your.html