Getting position and accounting data out of IB native python API

This is the final post. Not the final post of the blog; which may be good news or bad. But the final post in my short series on using the new native python API for interactive brokers. Having got some prices and submitted some orders we want to know whether we made any money or not; and what positions we have. Although the code is rather trivial, interpreting the results requires some understanding.

This is an update of the following post where I used the non native swigibpy (which I wrote nearly 3 years ago!).

This the fifth in a series of posts on using the native python API  You should read the first, second, third, and fourth. You should also read the documentation.

You need to get the code from this gist.

Getting positions

You will need to run the code in the last post first so you actually have a position to look at (this will submit a market order for Eurodollar futures). 

Now, if you've been following the series so far this kind of code pattern should be easy to follow:

## lets get positions

positions_list = app.get_current_positions()

In the client object:

def get_current_positions(self):

    ## Make a place to store the data we're going to return    

    positions_queue = finishableQueue(self.init_positions())

    ## ask for the data    

    self.reqPositions()

    <SNIP>

In the wrapper object:

def init_positions(self):
    positions_queue = self._my_positions = queue.Queue()

    return positions_queue

def position(self, account, contract, position,
             avgCost):

    ## uses a simple tuple, but you could do other, fancier, things here    

    position_object = (account, contract, position,
             avgCost)

    self._my_positions.put(position_object)

def positionEnd(self):
    ## overriden method
    self._my_positions.put(FINISHED)

Back in the client object:

def get_current_positions(self):

    <SNIP>
    ## poll until we get a termination or die of boredom    

    MAX_WAIT_SECONDS = 10    

    positions_list = positions_queue.get(timeout=MAX_WAIT_SECONDS)

    while self.wrapper.is_error():
        print(self.get_error())

    if positions_queue.timed_out():
        print("Exceeded maximum wait for wrapper to confirm finished whilst getting positions")

    return positions_list

Results:

print(positions_list)

 [
('DU15153', 
140109276672072: 56825063,GE,FUT,20181217,0.0,,2500,,,USD,GEZ8,GE,False,,combo:, 
10.0, 
244889.9)]

The results come back as a 4 tuple: account identifying name, contract, position and realised PNL (this might seem large - but this is a demo account so it will depend on what other people have been trading). I'd advise making a nice class to put these into, although I don't bother doing that here.

Finally be very careful about using these snapshots to work out what positions you have. Firstly it is common to get the position coming back repeated times. So you will need to filter out duplicates. Secondly positions may sometimes not appear at all.

I use my own record of trades to find out what my current position should be. I compare this to the IB snapshot throughout the day. If there are any breaks in the reconciliation I temporarily halt trading in the relevant contract until the break has gone away. Occasionally this is because IB has sent me a fill I haven't picked up, or because the IB position snapshot is missing some contracts.

Getting accounting data

The accounting data is a little more complicated. Firstly, we get three types of accounting data from IB, which need to be distinguished. Secondly, I choose to access the accounting data via a cache in the client object which only refreshes if the data we have is more than 5 minutes old. Of course once you understand the logic feel free to implement this differently yourself.

Accounting values

The first type of accounting data we get are accounting values (returned by updateAccountValue in the wrapper)

## get the account name from the position

## normally you would know your account but this is the demo

nameaccountName = positions_list[0][0]

## and accounting information

accounting_values = app.get_accounting_values(accountName)

In the client object:

def get_accounting_values(self, accountName):

    #All these functions follow the same pattern: check if stale, if not return cache, else update values
    return self._account_cache.get_updated_cache(accountName, ACCOUNT_VALUE_FLAG)

What is this doing under the hood? First we check to see if the cache is up to date (default is to check if we have got an account value in the last 5 minutes). If not then we ask for the accounting data to be updated. We then return the cached value (eithier historical, or just updated)

Let's have a look at the workhorse function that actually updates the data _update_accounting_data:

def _update_accounting_data(self, accountName):
    """    Update the accounting data in the cache
    :param accountName: account we want to get data for    

    :return: nothing    

    """
    ## Make a place to store the data we're going to return    

    accounting_queue = finishableQueue(self.init_accounts(accountName))

    ## ask for the data    

    self.reqAccountUpdates(True, accountName)

    <SNIP>

In the wrapper object:





## get accounting datadef init_accounts(self, accountName):
    accounting_queue = self._my_accounts[accountName] = queue.Queue()

    return accounting_queue


def updateAccountValue(self, key:str, val:str, currency:str,
                        accountName:str):

    ## use this to seperate out different account data    
    data = identifed_as(ACCOUNT_VALUE_FLAG, (key,val, currency))
    self._my_accounts[accountName].put(data)


def updatePortfolio(self, contract, position:float,
                    marketPrice:float, marketValue:float,
                    averageCost:float, unrealizedPNL:float,
                    realizedPNL:float, accountName:str):

    ## use this to seperate out different account data    
    data = identifed_as(ACCOUNT_UPDATE_FLAG, (contract, position, marketPrice, marketValue, averageCost,
                                      unrealizedPNL, realizedPNL))
    self._my_accounts[accountName].put(data)

def updateAccountTime(self, timeStamp:str):

    ## use this to seperate out different account data    
    data = identifed_as(ACCOUNT_TIME_FLAG, timeStamp)
    self._my_accounts[accountName].put(data)


def accountDownloadEnd(self, accountName:str):

    self._my_accounts[accountName].put(FINISHED)

The only novel thing here are these identifed_as(ACCOUNT_VALUE_FLAG, (key,val, currency)) guys. Because we push three different types of data on to the same queue we need to be able to distinguish them from each other. Rather than just push the raw data we're getting from IB (all stored as tuples, except for the single length timeStamp in updateAccountTime) we push an object which comes with a labelling flag to identify where it came from.

Back in the client object:

def _update_accounting_data(self, accountName):

    <SNIP>
    ## poll until we get a termination or die of boredom    

    MAX_WAIT_SECONDS = 10    

    accounting_list = accounting_queue.get(timeout=MAX_WAIT_SECONDS)

    while self.wrapper.is_error():
        print(self.get_error())

    if accounting_queue.timed_out():
        print("Exceeded maximum wait for wrapper to confirm finished whilst getting accounting data")



So far this is boilerplate, but this isn't: 


    # seperate things out, because this is one big queue of data with different things in it    

    accounting_list = list_of_identified_items(accounting_list)
    seperated_accounting_data = accounting_list.seperate_into_dict()



The queue is made up of three different kinds of data (values, updates, and time) from three different wrapper functions, so we need to split them up. We now have a dictionary, we now use this to update the different parts of the cache:


## update the cache with different elements
self._account_cache.update_cache(accountName, seperated_accounting_data)

We're now returned to the get_accounting_values function that will actually return the updated cache values we want:

def get_accounting_values(self, accountName):

    #All these functions follow the same pattern: check if stale, if not return cache, else update values
    return self._account_cache.get_updated_cache(accountName, ACCOUNT_VALUE_FLAG)

Results:

print(accounting_values)

[('AccountCode', 'DU15153', ''), ('AccountOrGroup', 'DU15153', 'BASE'), ('AccountOrGroup', 'DU15153', 'USD'), ('AccountReady', 'true', ''), .... <SNIP>

The account_value output will be a very long list of three tuples. Each one consists of (keyname, value, currency). Currency BASE indicates it is the accounts base currency (USD for the test account). Here are some of the more interesting account value entries.

• ExchangeRate: In the form of ('ExchangeRate', '1.00', 'USD') This is a very boring exchange rate as the base currency is USD.
•  FuturesPNL: This is how much we have made trading futures today, by currency. BASE is the total in base currency.
• StockMarketValue: Self explanatory
• NetLiquidation: This is what is my account worth if I close all my positions out (ignoring commissions and bid-ask). Its what I use to ultimately determine the total profit and loss used for rescaling my risk capital.
• CashBalance: Quite self explanatory. Negative means you have borrowed money. BASE is the net of your balances converted into base currency.

When you initially do a futures trade in a non BASE country you will have to borrow money for initial margin. Only if you move into profit beyond this will you have no net borrowing. IB charges you for borrowing money! This is done at LIBOR+spread so is expensive for currencies with higher interest rates (this spread is also why I use IMM's to get direct currency exposure). You can flatten out your exposure by doing a spot FX trade. Personally I try and keep a small positive balance in all currencies, although not excessive as this means I am taking on currency risk. Note you can't trade all possible pairs eg if you find you can't buy AUDGBP then try selling GBPAUD instead. The exception is Korea where you can't hold any speculative currency exposure i.e. not arising from margin requirements in other kinds of trading. All you are allowed to do is flatten your Korean currency position back to zero.

There are many more keywords than shown above. The best advice I have for trying to understand what they all mean is to start with a clean simulated account (a demo account is no good since you are exposed to other peoples random trading, and the account will often be 'empty'). Take a dump of the account value output, then do a trade. Take another dump, then wait for prices to move. By comparing each dump you should be able to see how account value, margin, cash and so on interact. This is quite time consuming but definitely worth it for an insight into how the IB accounts operate for the kind of instruments you want to trade.

Accounting updates

The next kind of data we get are accounting updates. I won't go through the code in detail, suffice to say it's very similar to accounting values, except that it's returning things populated by the updatePortfolio method in the wrapper; these are labelled in the joint queue with ACCOUNT_UPDATE_FLAG:

accounting_updates = app.get_accounting_updates(accountName)

print(accounting_updates)

[(140109274417416: 56825063,GE,FUT,20181217,0.0,0,2500,,GLOBEX,USD,GEZ8,GE,False,,combo:, 10.0, 97.93250275, 2448312.57, 244889.9, -586.43, 0.0)]

This is a tuple with (contract, position, marketPrice, marketValue, averageCost, unrealizedPNL, realizedPNL) A nice touch is to combine this with the information you get from get positions above: I leave this as an exercise to the reader!

Note: for completeness please note that you can also app.get_accounting_time_from_server(self, accountName) asdfto access the results from the wrapper method updateAccountTime although I don't bother myself. 

Anything missing?

I haven't covered the full gamut of what you can get from the IB API. There is a rich variety of data that is available. I haven't looked at news, fundamental data, option data, scanners and managed accounts because I don't use them. However I hope there is enough meat in this series of examples to get you started using IB with the new python API, and to avoid wasting time trying to understand some of the weirder behaviour.

This is the final of a series of five posts on constructing a simple interface in python to the IB API using swigiby. The first four posts are:


http://qoppac.blogspot.co.uk/2017/03/interactive-brokers-native-python-api.html
http://qoppac.blogspot.co.uk/2017/03/historic-data-from-native-ib-pyhon-api.html
http://qoppac.blogspot.co.uk/2017/03/streaming-market-data-from-native.html
http://qoppac.blogspot.co.uk/2017/03/placing-orders-in-native-python-ib-api.html


If you've found this series useful then your next steps might be to learn how to design a systematic trading strategy, use a python backtester to test your strategy, and to understand the nuts and bolts of creating a fully automated system.