Trading futures has one fairly substantial complication compared to many other assets. You can't just buy 'the CME Gold future'. There is no such thing. You need to select which delivery date future you are trading. Do you want to trade June 2015, July, August....?
Once you've chosen you can relax - but not for long. If you're trading June 2015 (I'm writing this in mid May), then in a few weeks time - unless you've given up on Gold and want to close your position - you'll need to get rid of your June future and buy another one. What's the best way of doing this?
This is the second post in a series giving pointers on building systematic trading systems (you don't need to read the
first; this post stands alone). However nearly all the content will be useful to non systematic futures traders. It might also help those who trade similar 'rollable' derivatives, such as spread bets.
First warning: This isn't an introductory guide to futures trading. I will be going very quickly or completely ignoring large parts of the mechanics of this asset class. You might want to read something first (
like this, other alternatives are
available).
Second warning: This is quite long. So you probably want to make coffee and / or visit the bathroom now.
Which contract to trade
First decision is which contract to trade. There are a number of things we need to think about.
Liquidity
The most important thing when deciding where on the curve to trade is liquidity. There are four main liquidity patterns that you'll see in the futures market. These patterns are very important for how we think about rolling these different markets, so it's worth spending some time understanding them.
Case 1 - Let's take it one year at a time
First let's have a look at Gold:
Source: CME
There's more after this, but I'm showing you the exciting bit. Focus on the '
total volume' and '
open interest - at close' columns*. Gold is an example of a contract with monthly deliveries, but where only selected rolls are liquid. We see a fairly typical pattern of liquidity declining over time, but for about a year there's enough of the stuff for most people.
*
I'm using these as a proxy for liquidity. I could also look at the inside size and depth of the order book; however these won't show what was actually executed - there can be a big difference in the two measures because of spoofing.
So unless you are a massive hedge fund for which only June is liquid enough you'll probably be able to trade the June, August, October or December 2015 contracts.
Contracts with a similar pattern (reasonably decent for 3 or 4 selected deliveries, or a year or so, with gradually declining liquidity) include other metals like platinum.
With liquidity declining monotonically every month, but again reasonably good for the next four deliveries and sometimes longer, there are the VIX and V2X equity volatility futures; and US (Henry Hub) natural Gas.
Case 2 - To infinity (almost) and beyond!
Here's another contract with plenty of possibilities, Eurodollar.
To avoid terminal boredom I won't keep going, but suffice to say there are 10K lots a day of volume available until March 2020. Again unless you're a massive hedge fund that is an awful lot of contracts to choose from.
Most short term interest rate futures, including LIBOR and EURIBOR, have these long tails of liquidity into the future, although none is quite as good as Eurodollar.
Case 3 - Tis' the season to be rolling...
A lot of commodity (agricultural, energy) contracts are strongly influenced by the season of their delivery month. This also has an impact on their liquidity patterns. Here is crude oil:
And some more (boring this isn't it?):
The pattern of good, but gradually declining, liquidity in a summer (June) and winter (December) contracts continues after this, with December showing particular strength as only December crude is quoted after 2019.
You'll see a similar seasonal pattern across many commodities, although the favored 'month' will vary, and few will have such long dated delivery months as crude. US Gas is an exception, as I've already noted.
Case 4 - You want the second contract? You can't handle the second contract!
Let's move on to our final example: US 10 year treasury futures please step forward.
No
I haven't missed anything out - that's your lot. All the liquidity is
in the front month. This thing rolls quarterly on IMM months, and a few
people have begun to move into September, but basically June 2015 is the
only game in town.
Other bond, IMM currency, and equity futures show this pattern - we can only trade the front month - there is no choice.
Cost & The Spread
To make a massive simplification trading cost depends mainly on the bid-ask spread, and on the volatility of the market. This is a simplification which is probably fine for a relatively small trader (larger traders will need to worry about the depth of the order book), who crosses the spread each time they trade (rather than using an
execution algo that could improve this price).
Lower volatility means we need to hold bigger positions to achieve a certain risk, which means higher costs. Since volatility is an important subject in its own right, I'll come back to it in a second.
Since we're making massive simplifications, let's make another one and say that the bid-ask spread will generally track the open interest and volume which we used to monitor liquidity above. So the most liquid deliveries should also be the cheapest to trade.
Volatility and kurtosis
I said above that low volatility means higher costs. It also means more leverage to hit a given risk target. Markets with low volatility also tend to have nasty kurtosis, as the low volatility has a nasty tendency of turning into high volatility when you're not looking.
Basically low volatility is pure evil. It is a devil-like* statistic, tempting you with it's promise of almost no risk, only to reveal it's true self - a horrifically ugly kurtotic distribution with a tails leading off into the far distance.
* God may not play dice. But the devil does. And he cheats.
So in case you haven't got the message: Do not trade low volatility delivery months!
Let's look at typical volatility term structure pattern*, for Eurodollar.
*
I am not talking specifically about realised or implied volatility here, since most will tend to show similar patterns.
In normal times nearer STIR contracts are the most volatile, with things getting less risky as you go further out in the term structure. They are constantly at risk of a central bank changing interest rates. This also means nasty kurtosis from those 25bp or 50bp jumps when an unexpected rate move happens. Generally you want to be near the front, but not too near so the kurtosis catches you.
But we don't live in normal times. When interest rates are basically zero, and expected to be that way for a while, the front contract doesn't move. Volatility then gradually picks up. Here are the prices of the first (blue) and ninth (red) Eurodollar contracts.
Source:
www.quandl.com
If you're colour blind, you can probably still take a good guess which is which.
The nicely wobbly line at the bottom is the ninth contract. Above it in blue you see an almost straight line which looks like it has been drawn freehand in Microsoft paint by someone with an almost impercetible nervous twitch? That's the front contract. I tried to measure its volatility but I got a floating point error.
Do you want to be holding this thing when
Janet finally
pulls the trigger?
The need to measure contango
A lot of traders, including myself use contango/rolldown/carry (pick your favourite nickname) to decide whether to be long or short. There are several ways of doing this (from best to worst):
- Compare current contract with the next nearer delivery. Assume we're holding September 2015, we'd compare it with August 2015 (monthly cycle) or June 2015 (quarterly).
- If there isn't a nearer delivery, compare current contract with spot. This isn't readily available for many instruments, although it is for equities and STIR.
- If there isn't a spot price available, compare current contract with the subsequent delivery. Assume we're holding June 2015, we'd compare it with July 2015 (monthly cycle) or September 2015 (quarterly).
This means it's better, if you have the option (so not the 'front only' case 4 markets), not to hold the very nearest contract as it will make it more difficult to measure rolldown.
The best rolldown is...
On a related subject it's best to be in the contract with the best rolldown for the sign of position you're holding - long or short.
* Since the sign of position you're holding depends on the rolldown be careful you don't get trapped into an endless alice in wonderland style decision loop.
If you're long then it's better to be in a delivery month which is backwardated versus the nearer month, i.e. the nearer price is above the price of what we're currently holding, so we'd expect the contract we hold to rally over time. If we can't hold a backwardated delivery then we'd want the month with the lowest contango.
If short you'll want to be in a month which is in contago versus it's nearer counterpart, i.e. the position month price is above the nearer month, and we'd expect it to fall as the contract ages. Again if we can't find a month which is in contango, then choose the least backwardated delivery.
Let's take an example. Here's the current term structure for VIX futures.
Let's assume we could hold any of these deliveries without liquidity issues, and we aren't worried about any other selection issues (probably not true, but bear with me). First of all suppose I wanted to be long VIX. None of the months shown above are in backwardation, so we'd settle for the least contangoed (is that a word?) which would be December, with contango of only 0.11 (comparing 18.51 to the nearer price of 18.40).
Now imagine we wanted to be short. The curve gets more contangoed as we get closer to the front. The best contango is clearly in June*.
* On this occasion we do have access to a spot price, but for simplicity I'm assuming that we can't measure the contango for the May delivery, even though this would probably be even better on this occasion.
Another example. VIX again, from a particularly dull week in the markets a few years ago.
If we wanted to be long we'd opt for the second contract (not shown, but it would be the November delivery), and if want to be short the fifth looks the best (backwardation of only 34.71 - 34. 32 = 0.24).
Price action
In certain contracts some traders argue that certain months show "better" behaviour than others. So for example if you're a trend follower you might look for a trendy month (whatever that is).
I'm not hugely convinced by this, but its certainly true that nearer months often do a different merry dance of prices than later ones, perhaps a tango rather than a waltz. So as systematic traders what we should be looking for is
consistent price behaviour.
It's better then, if you can, to stick to a contract which is of a roughly constant maturity. Right now for example I am the proud owner of four September 2018 Eurodollar futures, which expire in about 40 months. I'm currently thinking about rolling them to the December version, as I want to keep roughly that maturity.
Stick to a season
In a similar vein for seasonal commodities it probably makes sense (if you can) to stick to trading the winter or summer version (or spring, or autumn...). For example trading winter crude (as I do) implies holding December 2015, then moving imaginatively on to December 2016, before rolling into December 2017.... you get the idea.
One contract or two sir? (or three, or four...)
All of the above implies we're only going to hold positions in one contract at a time (except when we're rolling from one to another, but more of that in a moment). Why wouldn't we want to do this? Well normally delivery months of the same future are very highly correlated. So there isn't much diversification benefit from holding multiple dates.
If you're a small trader there is probably no value in using up your scarce risk capital to hold more than one month, unless of course you are explicatally trading spreads. If you have an enormous fund then you might have no choice but to spread your vast position over multiple delivery months.
If you're somewhere in the middle then a good place to start would be to trade different seasons of commodity contracts, like winter and summer crude, since they'll show the most differentiation.
Summary
Contract choice depends mainly on which kind of liquidity pattern we've got:
Case 1 and 2 (liquidity for a year, and "infinite" liquidity)
Avoid the first contract (can't measure rolldown). Avoid low volatility and evil skew contracts. Trade a liquid contract, with a tight bid-ask, with the best roll-down for the position you've got on. If you can, keep a roughly constant maturity, to get a consistent price pattern.
You might have to compromise on some of these points. Choosing the right contract is more of an art than a science.
Case 3 (seasonal)
As above, but if possible stick to one calendar month (eg December), then roll annually.
Case 4 (only the front)
We have no choice but to hold the first contract.
When to roll - the rolling period
Okay so we've managed to find which contract we'd like to trade. At some point however we're going to prefer another contract (or at least have no choice in the matter, because the bloody thing is about to expire).
All the considerations of
contract selection are also important when considering when to switch.
So we should only switch to contracts once they are liquid enough,
whilst their volatility isn't too low and have the right rolldown characteristics. We'd also want to switch before the current contract becomes illiquid, low volatility or when it's rolldown is no longer suitable.
This means that the decision of when to rolling can sometimes be a difficult balancing act. If you're long, and the next contract in line has less worse backwardation, then you might want to delay the roll as long as possible; whilst rolling before you end up holding the front contract, on which you can't measure rolldown and also has horrible volatility.
Rolling period
How long do you have to roll? It depends. If we're trading Eurodollar 40 months out, then we've got a maximum of three years to roll. Except given we want to remain about 40 months out, we've got roughly 3 months to do the roll.
Similarly if we're trying to stick to trading the second VIX future, then with monthly rolls we've got about four weeks, probably less.
On seasonal rolls, like winter December crude, we'll be able to roll once next years December contract is liquid enough. Ideally this would be before the current December becomes the front month, so we can still keep measuring rolldown, i.e. before the current November expires. Again this is usually a window of a few weeks or months.
For 'front only' markets like US 10 year futures the liquidity on the second contract might be available a few weeks in advance, or at the other extreme the next Korean bond future only becomes liquid around a day before the previous one expires.
Last possible moment
When do you absolutely have to roll? There is no easy answer. It might be just before the liquidity is completely sucked out of the contract you're currently in. It might be on the very day of expiry. Or, if you're long and unless you want to take physical delivery (and your broker will let you), before the first notice date where that is relevant.
Of course for cash settled contracts you don't
have to roll. If you know you're going to close your position anyway then it might make sense to save the execution cost of doing so if you're happy to take a small amount of risk on what the settlement price will be. I discuss this more below.
Price stitching
Let's have a look at VIX prices over the last few months. Assuming we wanted to stay a couple of contracts in the future we might have a picture something like this.
Ignoring the terrible excel date formatting you can hopefully see that at point 1 we switch from holding the May contract to the June. Then at point 2 we switch from the June to the July. If you mentally fill in the gaps you'll see that this naive series of price now contains two jumps, the second of which is more serious than the other.
There are various ways to deal with this problem. The first method, which I use, is probably the simplest and is known as the "Panama method". Essentially we keep the last July contract unchanged. We then shift the June contract up in parallel until it lines up exactly with the July on the roll date. The same process is applied to May. This gives us something like this:
The nasty jumps have gone - hurrah! However
- Drift effect: The little jumps in the price are effectively corrections for rolldown. By removing them we're including that rolldown in the price. This biases the trajectory of the price series (more here).
- Because of drift the price series we're using 20 years ago might have very different levels from what the actual price was at the time. If we go back far enough, we could even have a negative price.
- Percentage returns won't make sense any longer. If for a bond with price 120.0 we go back far enough to when the spliced price was 1.0, then a smallish single point move would be 100% of the spliced price.
I'm personally sanguine about these problems (I don't use percentage returns with the spliced price as the denominator, or actual prices in any trading rules, nor do I care whether a long term trend was partially caused by rolldown).
But some people prefer other stitching methods such as
proportional back adjustment (which also messes with price levels and volatility) or
linear weighting.
In the section below on detailed implementation I'll discuss how to put together historical series of panama stitched contracts. You should be able to adapt this procedure to other methods if you use them.
Trading the roll - let me count the ways
How do we actually go from holding May 2015 to June 2015, once we've decided that is the thing to do? I can think of four ways.
Passive
A very nice way to do the roll is
passively. Let's suppose we're long 5 contracts of May 2015 Gold, and we want to own June 2015.
Now suppose that we want to
buy two more Gold futures, for reasons that are not important. Should we buy May gold? No, that would be stupid. Instead of having to roll 5 contracts, we'd need to roll 7. Active rolling means trading, and trading costs money (though, as we'll discuss in a moment, often less than normal trading). Why do more than you have to?
Instead it makes more sense to buy 2 June 2015 contracts. We now own 5 May and 2 June. We still only have 5 May to roll.
Let's suppose that we've regretted our buying decision, and we now want to sell those 2 Gold we just bought. Does it make sense to sell the June? No, again that wouldn't be logical. If instead we sold the May then we'd only have to roll 3 May contracts.
Similar arguments apply to short positions.
To summarise:
Once the next market is liquid
If LONG, then BUY the next market and SELL the current one
If SHORT, then SELL the next market, and BUY the current one
Another way of thinking about this is you should always
add to your new position, and
reduce your current position.
Passive rolling can cut the separate cost of rolling to effectively nothing if you're trading a lot.
Disadvantages are that you need to have a reasonable window to roll in,
and if you're not trading much you probably won't be able to do all of
your roll in that window. So it's of little help if you're a slowish trader of those 'only the front' markets where we only have one or two days to roll.
If you can't passively roll before the 'I really need to roll now' moment, then you need to look at one of the active roll methods.
Spread trade
The classic way of actively rolling is to find somebody else with the opposite position, and trade the calendar spread with them. So I'd go to the market with my sell May buy June trade.
The advantage of this is you'd normally halve your execution cost (rather than paying a tick say on each leg, hopefully you'd only pay a single tick on the combo). You also have no price risk exposure on the roll trade compared to trading the legs separately.
The problem is that not all instruments have active, liquid, spread markets.
Separate legs
If you can't do a spread trade then you'll need to execute the two legs of your roll trade separately. If you have to cross the spread then this will cost you twice what a normal trade would cost.
There is also price risk involved if you don't cross the spread (and even hitting the bid/offer and paying the spread doesn't guarantee you won't still have price risk). Suppose you offer your five May gold contracts @1228.2 and bid for June@1227.6.
Then something bad happens and gold gets hammered. Your June bid is filled, but your May offer is now a hundred points off market. You now own ten Gold contracts - twice as many as you want to - in a falling market, and you've already taken a hundred point loss on them.
This isn't likely to end up being a very pleasant experience for anyone involved. So trading the legs should be avoided where possible.
Cash settle and rebuy
Very occasionally you might end up holding a contract to expiry. This is probably an absolute no-no if it requires physical settlement (unless you're a decent sized fund who deliberately arbitrages the settlement and has the merchandise to deliver, a warehouse to accept delivery, and the back office to handle the paperwork).
I did however accidentally take delivery on some currency IMM futures; that was cleared up with a few spot trades.
Clearing up after a delivery of Live cattle could be a lengthier and more problematic process.
For cash settled contracts it might not be the end of the world to hold at expiry.
I can think of times when it might make sense to cash settle. If you know you're not going to want to own the contract, and if volatility is low, then settling could be cheaper than selling out. It often makes sense to cash settle options trades which are well ITM or OTM, as the bid-ask to trade out can often be high just before expiry.
Even if you do want to own the contract after expiry then if the alternative is trading each leg separately then cash settling means you only have to pay execution costs on half of the trade.
Obvious disadvantages of cash settlement are that if you do want to maintain the exposure then you will have some price risk, as you're unlikely to be able to buy the new exposure at exactly the same relative level as where the settlement occured.
The double roll problem
From the discussion above its hopefully clear that a roll has completed when two events have taken place:
- We no longer have positions in the old delivery month
- We are using the new month price to generate our trading signals (having stitched the prices of the old month, and all its predecessors)
Now ideally these two things would happen at exactly the same time. In the real world
nothing ever happens at the same time. Also if we're passively rolling there will be a time when event 1 has only partly happened. Broadly then there are three options:
- Before we roll positions, we switch to using the new month price.
- During a passive position roll, we switch to using the new month price.
- After all positions are rolled, we switch to using the new month price after stitching.
The potential issue with the 'before' option is that the new month price might not be available, if for example you're dealing with a 'front only' market where the new month won't be liquid until the day of expiry (and you want to switch the price series over the previous night).
'During' has the nice property that if you're doing a gradual passive roll then you can switch over once you have a bigger position in the new month than in the old month. You'll need to pause any trading to do the switch.
The main downside of 'after' is you could be using the wrong price to generate trades (or worse still if the contract expires, you won't be seeing any updates to the price at all). It will be important to switch as soon as possible.
Different approaches clearly make sense for different instruments, and also different trading styles (a low frequency trader can be more relaxed about the whole thing). As we'll see below my own preference is to make 'after' a compulsory method, to simplify the control process for the roll.
Rolling in practice
Theory over; here are some details of how I manage the rolling process in practice.
I don't think you can fully
automate the process of rolling. It needs a lot of human judgement to
decide when to go. Although in theory this could be done systematically
it would need a lot of data, rules and special cases.
Also if I automated it I'd be worried that something would go horribly wrong...
So whilst everything is automated, in the sense I don't have to go and input orders myself, the computer won't start rolling until I've assessed it is the right time to do it.
Historical price stitching
If you're trading systematically then you'll need a history of prices to analyse before you start trading a new instrument; ideally tons to do backtesting, or at least enough to generate a signal on when you switch on your magic black box.
To start with lets assume we have downloaded a set of 'raw' prices for each of the contracts that cover our history, perhaps from somewhere like
www.quandl.com. You have also got the expiry date of each of those contracts.
So we'll have a set of these, here showing the Feb 1980 Gold contract:
run@bilbo ~/workspace/systematic_engine/sysdiag/scripts $ . displayraw LIVE GOLD 198002 20
PRICE ROWID
1980-01-29 702.0 10083
1980-01-30 658.0 10084
1980-01-31 681.5 10085
1980-02-01 683.3 10086
1980-02-04 660.0 10087
1980-02-05 687.0 10088
1980-02-06 713.0 10089
1980-02-07 689.5 10090
1980-02-08 694.5 10091
1980-02-11 702.0 10092
1980-02-12 697.5 10093
1980-02-13 688.0 10094
1980-02-14 670.5 10095
1980-02-15 663.0 10096
1980-02-19 632.0 10097
1980-02-20 661.5 10098
1980-02-21 656.5 10099
1980-02-22 615.0 10100
1980-02-25 648.9 10101
1980-02-26 625.5 10102
Decide your rolling policy
Depending on what kind of rolling pattern you have you need to determine your rolling policy. The rolling policy is made up of two components.
- How many days before expiry do we roll?
- Which subset of available contracts do we consider in our rolling cycle?
So for example take a case 1 contract (liquidity 'for a year') with monthly expiries, of which only the quarterly IMM dates are liquid, and where we want to avoid holding the front month perhaps so we can always measure rolldown. Then we'd want to roll something like 45 days before expiry (less than 31 means we'll end up in the undesirable front), and only include the HMUZ monthly expiries in our roll cycle.
For an 'infinite' liquidity like Eurodollar I like to stay about 40 months out. At this point on the curve there are only quarterly expiries anyway, and we'd want to roll about 1216 days before expiry.
If we're in an 'only the front' contract like Korean bonds then we'd
almost certainly include all contracts in the cycle. I'd probably want
to roll close to expiry, unless like US 10 year bonds liquidity comes in
earlier than that, or we're trying to avoid a first notice date.
In a seasonal case like Crude oil, since I'm sticking to the winter flavour, I'd only include the 'Z' contract in my roll cycle. I want to measure rolldown, and with monthly expiries I'll be doing so off the November 'X' contract, so I need to roll at least 45 days before expiry. Notice the items in red here.
run@bilbo ~/workspace/systematic_engine/sysdiag/scripts $ . showinstrument LIVE CRUDE_W
**************************************************************************************
Instrument code CRUDE_W, Light sweet crude Winter
rollcycle Z, pricecycle FGHJKMNQUVXZ, assetclass OilGas
exchange NYMEX, pointvalue 1000.000000 currency USD
lastrolloffset -60 days, firstrolloffset -90 days, carryoffset -1 priced contracts
IBSymbol CL
IB factor 1.000000 IB Price exchange NYMEX
Priced contract is 201512, forward is 201612, carry is 201511
Set your initial stitching dates
Using your rolling policy and the history of data you have got you need to determine your initial stitching dates. Start at the beginning of your history, and work out what contract you'd have been trading at that point. Then once you're N days before expiry, switch to the next contract in the subset; then repeat.
Let's take a simple example. Suppose I had Gold prices from 1st January 1980. To ensure I can measure rolldown I want to be in the second contract, but no further. My roll cycle is GJMQVZ which are the liquid months, all of which are around 2 months, 60 days, apart. To make things simple let's ignore the possibility of having a front month not in the roll cycle to measure rolldown off, and switch 70 days before expiry.
In January I'd want to be in April Gold, so I can measure rolldown off February. 70 days before April Gold expires I would then switch to June Gold. 70 days before June Gold expires I would move to August Gold, and so on.
I save down my stitching dates into .csv files so I can manually hack them later if required (see below).
Adjust your initial stitching dates
For a switch to happen there needs to be a closing price on the stitch for both the old and the new delivery months. If there isn't then the roll will fail. The initial stitching dates might need adjusting to ensure they are all good switch dates.
First I use an automated script which tries to move the stitching date backwards and forwards until it finds a date when everything matches up. This can have constraints on it like 'I never want to trade the first contract'.
If that doesn't work, and I still have bad dates then I start manually adjusting the dates, or taking other action. I might for example throw away earlier data if I can't find good roll dates.
If there is a gap in the new contract price then you could consider using the last known good price difference to extrapolate the price of the contract to the point when you can have a good roll date. This does mean your price series won't be pure, but could be viable for instruments where calendar spreads are very stable.
There is no one true set of stitching dates
If you're in a situation where you have some freedom with your stitching dates, bear in mind that due to liquidity pressures and arbitrage the roll spread can move quite a lot going into expiry, so you'd get different sizes of roll jumps baked into your price series. Changing your stiching dates will have a subtle effect on your price series and so on the results of any backtesting.
Create your stitched price series
Once all your roll dates are good you can create your stitched price series. You need to implement the algorithim of your choice to work backwards and repeatedly through the data.
A couple of technical points. Stitching can be done on the fly whenever you need a stitched price, or like me you can precalculate and update the stitched series whenever you get a new price for the current contract. The latter is faster, but does mean you are stuck with one set of stitching dates. You can also store your stitching dates seperately, or in the same table as the stitched price series.
Here is what my stitched price table looks like for the dutch equity market:
>>> atable=adjusted_price_table("LIVE", "AEX")
>>> atable.read_prices_for_code("AEX")[pd.datetime(2015,5,10):pd.datetime(2015,5,12)]
ADJ CARRY CARRY_CONTRACT FORWARD FORWARD_CONTRACT \
2015-05-11 08:01:14 489.675 NaN 201506 NaN 201506
2015-05-11 09:01:29 489.675 NaN 201506 NaN 201506
2015-05-11 10:02:52 487.850 NaN 201506 NaN 201506
2015-05-11 11:03:04 487.825 NaN 201506 NaN 201506
2015-05-11 12:04:23 487.950 NaN 201506 NaN 201506
2015-05-11 13:05:35 489.775 NaN 201506 NaN 201506
2015-05-11 14:13:16 489.225 NaN 201506 NaN 201506
2015-05-11 15:40:35 489.825 NaN 201506 NaN 201506
2015-05-11 15:40:36 489.825 NaN 201506 489.825 201506
2015-05-11 15:40:37 489.825 NaN 201507 NaN 201507
2015-05-11 16:40:55 490.600 NaN 201507 NaN 201507
2015-05-11 23:00:00 490.300 489 201507 489.000 201507
FX PRICE PRICE_CONTRACT ROW_IDS
2015-05-11 08:01:14 1.12494 490.175 201505 2618
2015-05-11 09:01:29 1.12494 490.175 201505 2619
2015-05-11 10:02:52 1.12494 488.350 201505 2620
2015-05-11 11:03:04 1.12494 488.325 201505 2621
2015-05-11 12:04:23 1.12494 488.450 201505 2622
2015-05-11 13:05:35 1.12494 490.275 201505 2623
2015-05-11 14:13:16 1.12494 489.725 201505 2624
2015-05-11 15:40:35 1.12494 490.325 201505 2625
2015-05-11 15:40:36 1.12494 490.325 201505 2626
2015-05-11 15:40:37 1.12494 489.825 201506 2627
2015-05-11 16:40:55 1.12494 490.600 201506 2628
2015-05-11 23:00:00 1.12494 490.300 201506 2629
I've highlighted the roll here in red. So before the roll (row 2625) we were trading May '15 as our current PRICE_CONTRACT. The next contract to roll into was June (FORWARD_CONTRACT). There is also a CARRY_CONTRACT which is what we use to measure rolldown from, and in this case it's the same as the forward (because AEX is a 'front only' instrument).
After we've rolled (row 2627) things have moved on, the PRICE_CONTRACT is now June, the FORWARD_CONTRACT is now July.
I'll return to the mechanics of the roll, and explain the rest of this dataframe, in a second.
States
At the kernel of how I manage rolls is the concept of an instrument roll state. Possible states are as follows:
- NOT rolling. We only have positions in the current delivery month. We're taking our price from the current month.
- PASSIVE rolling. We opportunistically buy or sell the current or new month to reduce our current position. We may have positions in the current or the new month. We're taking our price from the current month.
- IMMEDIATE rolling. We'll try and issue a spread order to do the entire roll. We may have positions in the current or the new month. We're taking our price from the current month.
- FORCELEG rolling. We'll try and issue seperate trades for each leg of the entire roll. We may have positions in the current or the new month. We're taking our price from the current month.
- FINISHED rolling. We have no positions in the current month. We're taking our price from the current month. Once we've switched our price feed, and done our stitching, we can put this back into a 'NOT' state.
This is simplified due to my insistence on only changing the source of prices after the roll of positions has been done. The state transition matrix is:
NOT -> PASSIVE
NOT -> IMMEDIATE
NOT -> FORCELEG
PASSIVE -> IMMEDIATE
PASSIVE -> FORCELEG
PASSIVE -> FINISHED
IMMEDIATE -> FORCELEG
IMMEDIATE -> FINISHED
FORCELEG -> IMMEDIATE
FORCELEG -> FINISHED
FINISHED -> NOT
Again you'd need more complex transitions if you allowed switching of the price feed to the new month at any point in the process, rather than after positions are rolled as I do.
I run checking reports which make sure that the positions we have are consistent with the state, and email me exceptions, such as a position in the next month when we're in a 'NOT' state, or a position in a 'bad' month which is neithier current or new.
Reporting
Although I don't fully automate rolls, I do use a number of reports to help me decide when to roll.
I generate this report 3 times a day:
Roll status with action needed
code
msg near_expiry_days position price_contract relative_volume
rollstatus suggest
28 VIX Roll
positions ongoing 8 -2 201506
0.276182 ALLOW CONTINUE
40 GAS_US
Roll positions ongoing 15 -1 201507
0.146196 ALLOW CONTINUE
42 EDOLLAR
Roll positions ongoing 8 4 201809
0.878177 ALLOW CONTINUE
20 AEX Roll near
on price, not liquid. 3 0 201505
0.004865 NOT START ROLL...NOTLIQUID
21 CAC Roll near
on price, not liquid. 3 0 201505
0.007042 NOT START ROLL...NOTLIQUID
Sorry about the wrapping. This report shows me:
- Futures that are currently rolling ('ALLOW') and whether the roll is passive ('ALLOW') or aggressive.
- Futures that are 'NOT' rolling, but which have less than 4 weeks before the point where I'd like to roll them.
It's
also possible to produce a similar table for all futures. Essentially the report works out what it thinks you should be doing ('suggest') and compares this with the current status.
The suggestion is based purely on volume and the number of days to the point where we'd normally want to roll, and does not take other liquidity measures into account, nor any other criteria. However it's a useful tool to keep track of markets for which a roll of one kind or another might be in order, or for which we need to step the rolling up a gear and go from passive to active rolling.
For example in the report above we see three markets on which we should CONTINUE rolling, and two where we would ideally START but the new month isn't yet liquid enough (volumes are less than 1% of the current month).
Before rolling I'd also look at a more detailed report for that instrument. Here's Australian STIR:
run@bilbo ~/workspace/systematic_engine/sysdiag/scripts $ . rollstatus LIVE AUSSTIR
For AUSSTIR current roll status is ALLOW; Suggested action is to CONTINUE
Roll positions ongoing
Roll positions ongoing
Contracts are
{'CARRY': '201603', 'FORWARD': '201609', 'PRICE': '201606'}
Positions are:
{'FORWARD': 0, 'PRICE': 8}
There are 387 days before price contract expires and 296 before carry
The suggested start roll period is 14.000000 days away
Volume data:
{'fiveday': {'201612': 2017, '201603': 10622, '201509': 14866, '201606': 7127, '201609': 4771, '201512': 18507, '201506': 8155}, 'oneday': {'201612': 841, '201603': 3809, '201509': 3171, '201606': 3200, '201609': 2197, '201512': 4740, '201506': 1018}, 'twoday': {'201612': 1637, '201603': 8801, '201509': 6979, '201606': 7518, '201609': 3880, '201512': 13166, '201506': 5006}, 'tenday': {'201612': 2691, '201603': 12441, '201509': 17871, '201606': 8721, '201609': 6231, '201512': 20925, '201506': 10516}}
As of dates:
{'FORWARD': Timestamp('2015-05-16 23:00:00'), 'PRICE': Timestamp('2015-05-16 23:00:00')}
Price matrix
201506 201509 201512 201603 201606 201609 201612
2015-05-16 23:00:00 97.87 97.93 97.97 97.96 97.910 97.84 97.76
Market liquidity (size, size, bid, ask) - priced contract
[586, 142, 97.87, 97.88]
Forward contract:
[1122, 275, 97.79, 97.80]
From this I can glean that I should ideally be done rolling within the next couple of weeks, which is over a year from the price expiry as befits an 'infinite' instrument. Looking at the liquidity picture, M16 averaged 8,155 contracts a day over the last five business days, and U16 did 4,771 which is less but still more than enough.
The new month ('forward contract') is trading with a 1 tick spread, in decent size.
I am long this thing to the tune of 8 contracts, so I want the best backwardation. The backwardation for M16 is 97.96 - 91.91 = 0.05 and for U16 is 97.91 - 91.84 = 0.07. So the backwardation will actually slightly improve when we switch. There is no reason not to do this roll.
Rolling execution algo
My execution process will behave differently according to the roll state:
- NOT rolling. Buy and sell only the current month.
- PASSIVE rolling. Opportunistically buy or sell the current or new month, as above. When (if) we have no positions left in the current month change state to FINISHED.
- IMMEDIATE rolling. Issue a spread order to do the entire roll. When (if) completely filled change status to FINISHED.
- FORCELEG rolling. Issue
seperate trades for each leg of the entire roll, assuming both meet certain liquidity requirements. When (if) completely filled change status to FINISHED.
- FINISHED rolling. Buy and sell only the new month.
Live price stitching
Let's return to the extract of stitched price data for AEX that we looked at above. However I am now going to wind the clock back to just before the roll began:
>>> atable=adjusted_price_table("LIVE", "AEX")
>>> atable.read_prices_for_code("AEX")[pd.datetime(2015,5,10):pd.datetime(2015,5,12)]
ADJ CARRY CARRY_CONTRACT FORWARD FORWARD_CONTRACT \
2015-05-08 23:00:00 488.750 488.25 201506 488.250 201506
2015-05-11 08:01:14 490.175 NaN 201506 NaN 201506
2015-05-11 09:01:29 490.175 NaN 201506 NaN 201506
2015-05-11 10:02:52 488.350 NaN 201506 NaN 201506
2015-05-11 11:03:04 488.325 NaN 201506 NaN 201506
2015-05-11 12:04:23 488.450 NaN 201506 NaN 201506
2015-05-11 13:05:35 489.275 NaN 201506 NaN 201506
2015-05-11 14:13:16 489.725 NaN 201506 NaN 201506
2015-05-11 15:40:35 490.325 NaN 201506 NaN 201506
FX PRICE PRICE_CONTRACT ROW_IDS
2015-05-08 23:00:00 1.12964 488.750 201505 2617
2015-05-11 08:01:14 1.12494 490.175 201505 2618
2015-05-11 09:01:29 1.12494 490.175 201505 2619
2015-05-11 10:02:52 1.12494 488.350 201505 2620
2015-05-11 11:03:04 1.12494 488.325 201505 2621
2015-05-11 12:04:23 1.12494 488.450 201505 2622
2015-05-11 13:05:35 1.12494 490.275 201505 2623
2015-05-11 14:13:16 1.12494 489.725 201505 2624
2015-05-11 15:40:35 1.12494 490.325 201505 2625
Focus on the red row. At this point we've done our position rolling, and we're ready to move from collecting K16 for pricing ('PRICE CONTRACT') to collecting M16 ('FORWARD CONTRACT').
Remember to do a roll we need to have a price in both the old and new contracts. But there is no FORWARD price, since I'm only collecting end of day. In this common case I offer the user the option to extrapolate from the last good price difference, or to put their own forward price in.
In row 2617 the last good price difference was 0.50 (488.75 versus 488.25). So the extrapolated forward price is 490.335 - 0.50 = 489.825. We add this as a pseudo row:
ADJ CARRY CARRY_CONTRACT FORWARD FORWARD_CONTRACT \
2015-05-08 23:00:00 488.750 488.25 201506 488.250 201506
2015-05-11 08:01:14 490.175 NaN 201506 NaN 201506
... removed ...
2015-05-11 14:13:16 489.725 NaN 201506 NaN 201506
2015-05-11 15:40:35 490.325 NaN 201506 NaN 201506
2015-05-11 15:40:36 490.325 NaN 201506 489.825 201506
FX PRICE PRICE_CONTRACT ROW_IDS
2015-05-08 23:00:00 1.12964 488.750 201505 2617
2015-05-11 08:01:14 1.12494 490.175 201505 2618
... removed ...
2015-05-11 14:13:16 1.12494 489.725 201505 2624
2015-05-11 15:40:35 1.12494 490.325 201505 2625
2015-05-11 15:40:36 1.12494 490.325 201505 2626
Notice this is a clone of the previous row, apart from the imputed forward price (in bold) and the slightly incremented date (to respect the unique datetime key in the table).
This works well for me, although for high frequency you'd need to use a smaller date increment, and you might want to mark the row with a flag to indicate it's not a true price.
Now for the actual roll. First of all we're going to add another pseudo row.
ADJ CARRY CARRY_CONTRACT FORWARD FORWARD_CONTRACT \
2015-05-08 23:00:00 488.750 488.25 201506 488.250 201506
2015-05-11 08:01:14 490.175 NaN 201506 NaN 201506
... removed ...
2015-05-11 14:13:16 489.725 NaN 201506 NaN 201506
2015-05-11 15:40:35 490.325 NaN 201506 NaN 201506
2015-05-11 15:40:36 490.325 NaN 201506 489.825 201506
2015-05-11 15:40:37 489.825 NaN 201507 NaN 201507
FX PRICE PRICE_CONTRACT ROW_IDS
2015-05-08 23:00:00 1.12964 488.750 201505 2617
2015-05-11 08:01:14 1.12494 490.175 201505 2618
... removed ...
2015-05-11 14:13:16 1.12494 489.725 201505 2624
2015-05-11 15:40:35 1.12494 490.325 201505 2625
2015-05-11 15:40:36 1.12494 490.325 201505 2626
2015-05-11 15:40:37 1.12494 489.825 201506 2627
Notice that the new PRICE is the same as the FORWARD in the prior row 2626, because M16 is now the PRICE_CONTRACT. N16 is now the FORWARD_CONTRACT, for which we don't yet have a valid price. The ADJ price is the same as the PRICE; this will always be the case in this data frame
Finally we need to resplice the adjusted price series. Since there has been no real price movement between rows 2626 and 2627 (and also the previous row, although this isn't always the case) the ADJ should be the same in both cases. To achieve this the ADJ in row 2626 needs to be 0.50 lower. We also need to move
all previous ADJ right back to the start of our data history down by 0.50.
Here is the final result, with the new ADJ highlighted in bold, all of which are 0.50 less than the previous entries.
ADJ CARRY CARRY_CONTRACT FORWARD FORWARD_CONTRACT \
2015-05-08 23:00:00 489.250 488.25 201506 488.250 201506 2015-05-11 08:01:14 489.675 NaN 201506 NaN 201506
...removed ...
2015-05-11 14:13:16 489.225 NaN 201506 NaN 201506
2015-05-11 15:40:35 489.825 NaN 201506 NaN 201506
2015-05-11 15:40:36 489.825 NaN 201506 489.825 201506
2015-05-11 15:40:37 489.825 NaN 201507 NaN 201507
FX PRICE PRICE_CONTRACT ROW_IDS 2015-05-08 23:00:00 1.12964 488.750 201505 2617
2015-05-11 08:01:14 1.12494 490.175 201505 2618
.... removed ...
2015-05-11 14:13:16 1.12494 489.725 201505 2624
2015-05-11 15:40:35 1.12494 490.325 201505 2625
2015-05-11 15:40:36 1.12494 490.325 201505 2626
2015-05-11 15:40:37 1.12494 489.825 201506 2627
This re-adjustment of all previous rows in the table can take a while, during which the database is locked. I need to pause my trading and sampling processes (process control will form another post in this series) to make sure they don't try and access the price data.
From a technical point of view I've found that partitioning my spliced price data so that each instrument occupies one file on disk is more efficient (the readjustment is quicker; in the order of five minutes for 30 years of history rather than hours) as well as isolating so only a single file is locked at a time, and all other instruments are unaffected.
Fin
That is pretty much everything you need to know about futures rolling, or at least everything I can think of right now. However if I've missed anything out, or something is unclear, then comment below or email me.
This is the second in a series of posts on building systematic futures trading systems.
The first post is
here.
The next post is
here.
