Tuesday 21 July 2015

Systems building - execution

People often get systematic and automated trading mixed up. The latter is a subset of the first. You can't have a system which is fully automated if it relies on discretionary input, no matter how small. But you can have a system which needs a human to make it run, even though there is no discretion, and its fully systematic.

The main area where humans are often used with systematic trading is in execution. It's relatively common for people to create software that analyses prices and produces a trade, which the person then has to execute.

(At the large hedge fund I used to work for, we used a mixture of automated execution algos, and real people to execute trades. FWIW the best humans were better than the best algos, on certain kinds of markets)

This post isn't for you. It's only for those who want to automate their execution as well, and avoid the stress of actually trading.

If you don't want to be like this guy, then read the rest of this post.

This is the fourth post in a series giving pointers on the nuts and bolts of building systematic trading systems. You may also want to look at my post on building a simple execution algo, which I'll refer to occasionally here.

Event driven versus Sample driven

As I've discussed before in this series there are two ways of running a trading system. The first method is event driven; you wait for something to happen (price hitting a particular level, or the next tick arriving) and then you take the appropriate action (buying or selling, or cancelling an existing order perhaps).

The second method, which I use, is sample driven (and there is no agreed nomenclature here, so you may hear it called other things). Here we periodically check / 'sample' the data we are interested in (eg prices), calculate what we want our optimal position to be, and then trade to achieve it.

The former system is most suited to high frequency trading, and to relatively simple algorithims (if you have to spend a minute analysing the tick event, then you lose any benefit from running an event driven system). It doesn't work as well with, for example, trading relative value systems.

Anyway this isn't the place to get into a debate about which is better. For my application (relatively slow trading) sample driven trading is more natural. There are also advantages in that debugging is easier, and code can be simpler.

Much of this post will be relevant to event driven systems, but those running sample systems will get more out of it.

Key principles

There are a number of factors, some mutually exclusive, to consider when designing a system to do execution. In particular there is a potential conflict between robustness and speed. To be precise, if I'm in a situation where the system might do something potentially dangerous such as an incorrect trade, I would rather delay the trade until the situation can be resolved; even if the delay could be several days.

Because I want a fully automated system, that literally should be able to run for months without being touched or watched (rolls apart), this robustness is incredibly important.

The optimal position

To be clear then before the part of my program that does execution comes into the picture something else has created a theoretically optimal position, which was true at some point in time, and with some data (in practice then, for my simple technical system, at some price).

I'll talk more in a later post about whether this is a separate process, or all one big happy function, and how separate processes 'talk' to each other.

Essentially then the job of the execution component of my system is to compare the optimal position with the current position; and if they are out of line, try and do a trade that brings the actual position back to what we want it to be.

If you want more detail on what I mean by 'out of line' you'll have to read my forthcoming book, www.systematictrading.org, chapter 11.

What can go wrong

This all sounds very simple, but in fact there are a number of things that can go wrong:

  • Optimal position is  unavailable
  • The price on which the optimal is predicated is out of date
  • We don't know what our current position is
  • Our current position is out of date, as we've recently done a trade that will change it
  • We're running two processes and they will both issue the same order, ending up with the position 'overshooting' the optimal
  • The market is closed
  • The market isn't liquid enough to do the adjusting trade required
  • We might, for some reason, not want to do the adjusting trade

Writing exhaustive lists of 'what can go wrong' like this is vital before you write a robust process. We need to make sure all our processes deal with these issues. Also my bias towards purely automated systems that run themselves, on which a delay in trading can be shrugged off, influences how these problems are solved.

Getting the current position: Storage and lining up

I won't discuss where the optimal position is stored, as this will be the subject of a future post. Just for now imagine there is a database table populated with it (as indeed is the case in my own system). However just to note that if we can't read the optimal position then we probably have a fairly broken database (or perhaps a file lock), and we shouldn't try and do any trading.

There are two main places you can store your current position:

  • Your own database
  • Your brokers database
In the latter case you'd need to use the brokers API to get positions. Note that unfortunately neithier of these methods is 100% reliable*. I've had issues the IB API** giving me the wrong position (usually eithier missing one instrument entirely, or doubling up on the real position). I've also had issues with fills arriving which I've missed, thus making my own database estimate of the current position wrong.

* The NASA way of avoiding conflicts of this kind would be to have three sources of current position and institute some kind of voting algorithm. Sadly I can't think of a further, independent, source of position.

** References to my broker IB (Interactive Brokers), can clearly refer to other brokers as well.

The solution I use is to make my database the primary source of current position. I then crosscheck this 3 times a day with the position the broker thinks I have. If there is a difference of opinion between me and the broker then I change the trading state to STOP (see more below about different states). Genuine differences of opinion are rare, and more often than not caused by a position cash settling or being forced to close by IB when for some reason I haven't closed it myself before expiry (discussed more here).

I also avoid creating new orders when I think, or know, I have an existing order (of which more below).

Lessons from the database managers

Let's take a few of those problems I outlined above and rephrase them slightly:

  • Our current record is out of date, as we've recently done a transaction that will change it
  • We're running two processes and they will both issue the same transaction, ending up with the record being incorrect.

These problems then are exactly those faced by people who create database systems. The solution that they, and anyone involved in worrying about files being updated by multiple users / processes, is locking.

If I google (images) 'database' this is the first thing that comes up. It would be much cooler if databases actually came with holograms of pieces of paper flying in and out of your server.

I use locking in the following way. Locks are applied at an instrument level (eg to the whole of the Gold future, regardless of which contract we're trading). The normal status of an instrument is unlocked. This means we can issue a new order related to that instrument. Once an order is issued however the instrument becomes locked.

In a locked status no new orders can be issued for an instrument. Because I am super paranoid I'll also check with the IB API that there are no outstanding orders; even if the instrument is unlocked if the API reports the existence of an outstanding order (which shouldn't happen... but...) means I can't do any more.

The lock will only be removed once the order is completely filled and the current position updated to reflect that, or the order is cancelled by my software (of which more later). Because I can miss fills, and because of other gremlins, I have a periodic (hourly) process which checks to see if there are any instruments that are locked, but with no outstanding orders reported by the IB API. If there are, then they are automatically unlocked.

This may seem like paranoia. Why not just check for oustanding orders from the IB API, and don't trade if they exist? Because there is a tiny period of time where IB will say 'yeah, no outstanding orders' but the relevant fill hasn't yet arrived (or the fill may be missed somehow) and my current position hasn't been updated, meaning I might try and issue the same trade again. By periodically removing locks that apparently aren't needed I do open myself up to this happening, but it's much less likely than if I used the presence of IB outstanding orders as my only check to see if it is safe to trade.

Enemy of the state

I won't always want to trade an instrument, or do certain types of trade. For this reason I have a number of trading states which an instrument can be in:

  • OK - allow all kinds of trades
  • NOOPEN - do not open any new trades, but allow any existing orders to continue.
  • NOINCREASE - do not do any trades that will increase our position, but allow closing trades. I use this a lot; for example when I recently wanted to get out of my German 2 year bond position.
  • NOCLOSE - do not do any trades that will reduce our position. It's hard to think of a situation where this would make sense, but I hate asymmetry.
  • STOP - do not open any new trades, and cancel any existing orders. Once existing orders are cancelled, this migrates to 'NOOPEN'
  • CLOSE - immediately try and issue a trade to close the entire position (but which will respect trading limits; see below).

Locks and states are different; locks are applied and can be removed by software. The state can be changed by software (such as when opinions on current positions vary); but can only be reset to 'ok' by human hands.

Rolling rolling rolling...

Applicable to future traders only (or spread bets, options... others where positions can't be held indefinitely and need to be 'rolled over' to the next maturity). You should have already read my post on the mechanics of rolling. Suffice to say that we can be:

  • Not rolling, in which case we'll buy and sell the current contract
  • Passively rolling, when we'll issue reducing orders in the current contract, and increasing orders in the new one
  • Actively rolling with a spread trade, when we'll issue a single order to (if long) sell the current contract and buy the new one
  • Actively rolling with an individual leg trade, when we'll issue two orders to close the current contract, and open a position in the new one

The important point here is that the code first needs to identify the roll state, work out which combination of orders makes sense, and then needs to have the logic to issue one or two orders in the current and/or new contract; or a spread order.

Open all hours

There is no point trying to trade if nobody wants to trade with you. There are five reasons I can think of why you might not be able to trade:

  • The market is closed today (a regular closure, depending on the day)
  • The market isn't open right now (a regular closure, depending on the hour).
  • The market is unexpectedly closed.
  • There isn't enough liquidity in the market to trade
  • You have been banned from the market, or blown up and lost all your money; or upset your broker.
Don't copy this guy if you don't want to be banned from the market

Leaving aside the last point; for the first two points we can of course download and create tables storing market holidays and market opening times. Beware this is a lot of work; and you need to then waste your life worrying about daylight savings changes. This doesn't deal with point three of course. 

For point four you're going to have to check the liquidity before you trade. I'll discuss below exactly what a liquidity check might involve.

I pretty much choose the laziest possible option; which is to treat the first four points all in the same way as a lack of liquidity. What I do in practice then is have a table of when I know markets should be open and liquid(ish). During the times specified in that table if I feel the need to trade I will periodically check to see if that market is liquid. If it is, then I go ahead and trade (if trading is needed). If not, then I don't. Not liquid could mean the market is on holiday today, or has closed early for some reason, or it might just not be liquid.

Note that if I was even lazier, and just checked all markets 'all'* the time, then I'd be frequently checking a lot of closed markets which is a waste of processing power and bandwidth.

* My execution process runs from 10pm to 8pm the next day, local UK time. It is kicked off Sunday evening, Monday evening .... Thursday evening. The two hours of down time is used to run reports and create backups.

Liquid liquid everywhere and not a drop to drink

I said above we need to check for liquidity. There are many ways to measure liquidity of course. In my simple world, where I only have level 1 data (and L2 in any case could be corrupted by spoofers), I use the following simple test:

  • Is the inside bid-ask spread in the market smaller than X (where X is some value I've set for each market, which depends on how I'm executing the order)?
  • Is the size available at the bid and ask larger than the trade I want to do?

Note that if the market is closed then the values for bid-ask spread and size will be NAN and NAN respectively, so by definition there isn't enough liquidity.

I'll discuss in a second how the method of execution affects the required maximum spread. Just to note that if the size available is smaller than the trade I want to do (but is bigger than zero), and I want to do more than one contract, the execution code will reduce the size of the trade so its possible given the available size.

There is a further check I will do at this stage. Remember from above the theoretically optimal position, which was true at some point in time, and with ... some price. So I need to have a concept of staleness, and a concept of maximum price movement.

To put it another way, given the speed of my system, I would be concerned if I'd calculated my optimal position based on a price from more than a hour ago. By the way this will happen if you're running seperate optimal position calculation and trading processes as I am; and the latter kicks off in the morning before the former has had a chance to update the price. It won't happen if you have a single pipeline process 'collect price -> calculate optimal position -> create order', or if you're using event driven systems. Again this kind of stuff will be the subject of another post.

Secondly I'd be concerned if the price had moved since I checked it. If for example a market had moved more than 4 daily standard deviations in price in the hour or less since I last grabbed a price, then it's quite likely my optimal position is out of date, and I will want to recalculate it before trading. Otherwise the risk is I might buy based on the old price, and then quickly sell again, once a new price has percolated through the system.

The messy business of actually executing

When you actually execute you can:

  • Issue a market order, that will most probably cross the spread
  • Use your own algo, as I do (much, much more is here).
  • Use a brokers algo. IB have a good set of algos for equities, fewer for futures.
  • Use a third party algo. Unfortunately the good ones are only available to institutional traders. I can recommend QB for interest rate products.

I will usually use my own algo, although for manual trades (see later) I give myself the option of using a market order. Eventually I'd expect to have a suite of algos for different instruments; so I have the ability to do this ready in my database, although I don't yet use it. Algo evaluation is a science; just like evaluating other trading systems it has very noisy data and again only large institutions can realistically collect enough data to make meaningful comparisions (and possibly not even then).

As I alluded to above I have different liquidity requirements depending on whether I am issuing a market order or using my algo. If I'm issuing a market order I want the spread to be tight (normally a single tick, or whatever the minimum increment is, unless the market normally trades a little wider), since I know for sure I'll be paying half of it. If I'm using my algo I am less choosy; a much larger spread might mean I capture more of it when I submit my initial lot at the bid (if I'm buying) or offer (selling). This is true to a point - very large spreads mean the market is probably dangerously thin and I wouldn't try and issue an order.

Safety first

The scariest thing about fully automated trading is the doomsday scenario; what if this thing just goes bad and trades like crazy?

What my automated system might say if I tried to stop it trading. Although, just to be clear, my name isn't Dave.

Although I've been ultra careful there is still an outside chance something could go wrong. To avoid this I use daily, and 'lifetime' trade limits.

So before I actually issue an order the software will check to see how many contracts I'd done today already, and whether I will break that cumulative limit. If so it will reduce the trade (sometimes down to zero). I also have 'lifetime' limits. In practice if I'm going on holiday I will set the lifetime limits to a reasonably large level of trading I'd expect to do over that period (clearly less than N x daily limit, where N is the number of days I'm away, otherwise they won't bind), and then when I get back I'll reset them to large numbers.

Limits are set by looking at values in my backtest, and so are contract dependent.

In my current setup active roll trades (as spreads or legs) are exempt from trade limits, although once done they count towards the limit (often meaning you can't do any more trading that day after a roll). It would probably be better to count rolls seperately towards another, larger, limit. That is on my, very long, to-do list.

Note that IB API also has limits, but they are not instrument dependent. Unless you're trading only stocks, or  just one future, this makes them pretty useless. So I turn them off.

Correcting mistakes

Stuff happens. And when it happens you need to fix the mess it creates.

The first kind of stuff, which I've already mentioned, is where my position and the IB API position get out of line. Normally what has happened here is that I issued an order which was filled, and I missed seeing the fill; or IB closed out or settled a position which I hadn't got around to rolling in time.

To deal with this my software has the ability to do a manual fill or a balancing order. The former is where I update an existing, unfilled, order in my database to include the fill that actually happened. The latter is where I create an entirely new order in the database to reflect the trade that happened. After I've done this I should be in a situation where my, and my brokers, record of position are perfectly matched.

The other kind of stuff is sometimes I might want to trade outside of the normal execution process. For example suppose I have a contract I have to roll today. I've tried passive rolling, issuing spread orders, and even individual leg orders. None of these have filled, so what I will do is issue a manual trade from the command line, and then manually override the normal execution method to do a market order. Unlike a manual fill or a balancing order a manual trade will be sent to IB for execution, as well as going in the database.


Hurray. Instead of being tied to your computer, following its robotic commands to execute trades, you can now do something else instead.

In the next post I'll look at working out how much lovely moolah you're making.

This is the fourth post in a series on building systematic trading systems.

The first three posts are:


The next post is here:



  1. Impressive process! Thanks for the detail. Always interesting to see what others are doing.
    Are you not concerned with slippage when entering market orders? The DAX future is a pig for poor fills using market orders and I have found that you can finesse away the slippage with manual limit orders. Of course you could be making so much ‘moolah’ that slippage costs are not an issue :-) 

  2. Hi Ryan. You're absolutely correct. I don't use market orders unless I have to (like for example the 'must roll' situation mentioned above). The default for all markets is to use my home made execution algo (http://qoppac.blogspot.co.uk/2014/10/the-worlds-simplest-execution-algorithim.html). That cuts my slippage by around 90% from what just submitting a market order would.

    Note that in my backtest (http://qoppac.blogspot.co.uk/2015/03/simulating-my-futures-system.html) I conservatively assume I am paying half the spread from issuing market orders, equating to 0.06SR in costs (this also includes commissions) or 1.5% a year.

  3. Without seeing all the code I'm trying to understand how some of the logic is built into this.

    Lets say your rolling logic is that would like to be completely rolled into the next contract by the 3rd calendar day before first notice. Then, say you have a "grace" period of 5-days where if you need to buy/sell a new contract and its within 5-days of your full roll date you just move to the next contract instead of executing in the current contract because its rolling soon.

    I'm running through all the logic and scenarios that need to be coded for this to be completely automated:

    First scenario: Lets say your long 5 June S&Ps, its within the grace period, and you need to increase your position(more long). In this case, you would simply buy x-contracts in the Sep contract. So now your long 5 Junes and x Seps and your total position needs to coded so it is now the aggregate of the two contracts.

    Second scenario: if your long 5 June S&Ps and its within the grace period and you need to decrease your position(still long but less so). You would then sell x June contracts.

    Third scenario: your long 5 June S&Ps, its within the grace period and your optimal position is to now be short. So programmatically, you need to sell 5 June S&Ps and then sell x Sep S&Ps

    The logic that needs to be coded is:
    - When your in multiple contracts of the same market they need to be aggregated somehow despite different tickers.
    - When a change to a position is in the same direction as your current position and its within the grace period you buy/sell the next contract.
    - When a change to a position is in the opposite direction you want to close positions in the current contract.
    - The third scenario I'm still trying to figure out.

    Am I thinking of this all correctly in terms of programming logic? Is there a simpler way to do all of this? I obviously can't see your execution code so I'm trying to create a layout in my mind to get going.

    Any thoughts more than helpful.

    1. Yes the secret is to have all your code written to think in terms of an instrument, whose exact representation is unknown until the point of trading.

      Then, this is the code I use:


  4. I'm sure you have experience with cross-sectional monthly rebalanced strategies. I've come across a tricky situation involving cross-sectional strategy and execution with trading instruments across timezones.

    Because of the mismatch in time zones in certain markets(which obviously makes executing and ranking cross-sectionally tricky) do you recommend waiting a day to execute? For example, if I calculate the signal on 4/27, execute on the signal on 4/28? What has your experience been with this mismatch?

    Thank you.

    1. It depends on how frequently the underlying data is updated. If you're (say) using daily closing prices to calculate monthly returns as an input into your signal, and trading monthly, then there is almost no point in waiting. But on the other hand if you're waiting for some economic data that is released monthly on the 5th in every country, and you trade prematurely with numbers that for some country are a month out of date, then this is a problem and you should delay.


Comments are moderated. So there will be a delay before they are published. Don't bother with spam, it wastes your time and mine.