tag:blogger.com,1999:blog-261139923818144971.comments2018-08-17T05:59:56.434+01:00Investment IdiocyRob Carvernoreply@blogger.comBlogger1441125tag:blogger.com,1999:blog-261139923818144971.post-14057174421742448312018-05-18T06:43:58.141+01:002018-05-18T06:43:58.141+01:00A few reasons: (a) I don't have OHLC data in m...A few reasons: (a) I don't have OHLC data in my current database, although I could easily take the first sample each day as open, (b) It's more conservative, if the market is fast moving you might not get the open price. I suppose a better measure might be the average of close and open on the following day (close to VWAP).Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-27845001925228827082018-05-17T20:43:32.990+01:002018-05-17T20:43:32.990+01:00Hi Rob, an older post but hoping you don’t mind a ...Hi Rob, an older post but hoping you don’t mind a new question on this. <br /><br />I’m intrigued why you backtest with a one day lag at the closing price, rather than just using the open price following a signal at close? <br /><br />Thanks - big fan of your books - looking forward to the next one? :) <br /><br />David david leakehttps://www.blogger.com/profile/16887600679285433839noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-61884661340755442142018-05-16T16:10:55.964+01:002018-05-16T16:10:55.964+01:00Ok I worked out the problem. Your derivation is on...Ok I worked out the problem. Your derivation is only applicable if there is unlimited leverage and short selling is allowed to achieve a particular risk target, since you don't account for the constraints in the portfolio (positive weights, summing to 1). In that case the maximum SR = maximum geometric mean (indeed with unlimited leverage maximum SR = maximum portfolio mean as well). Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-80609107971620230452018-05-16T16:04:23.824+01:002018-05-16T16:04:23.824+01:001. Find w that maximizes GM = w'r - 0.5(w'...1. Find w that maximizes GM = w'r - 0.5(w'*C*w) Taking derivatives and solwing for w(i) giwes me optimal w = C^(-1)*r<br /><br />2. Find w that maximizes SR = w'r / sqrt(w'*C*w) gives me the same optimal w = C^(-1)*r<br /><br />I used arithmetic mean in 2, should i use GM? P_Serhttps://www.blogger.com/profile/00382284297691291199noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-29395700217982287352018-05-15T18:38:54.169+01:002018-05-15T18:38:54.169+01:00If we take a specific example from my book; in cha...If we take a specific example from my book; in chapter five the max GM portfolio is about 10-20% in bonds, but the max SR portfolio is about 60% bonds.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-2231303665509308842018-05-15T18:37:57.710+01:002018-05-15T18:37:57.710+01:00Your algebra is wrong. Max GM is indeed Kelly opti...Your algebra is wrong. Max GM is indeed Kelly optimal, but neither is max SR. GM = m - 0.5s^2 (good enough approximation) but SR = m/s (where m is the excess return, after deducting risk free rate; and s is the standard deviation). Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-16611079169254596912018-05-15T18:26:56.355+01:002018-05-15T18:26:56.355+01:00Dear Rob, I have a question after reading your sec...Dear Rob, I have a question after reading your second book, and I don't know which post it belongs to, so i decided to write it here: <br /><br />In the book you distinguish between maximum GM and maximum Sharpe Ratio portfolios, this is a bit confusing to me, as I always thought that maximising Expected geometric mean of portfolio gives you Kelly-optimal allocation, which in turn maximizes Sharpe Ratio of the portfolio (this can be proven mathematically using linear algebra). Could you please elaborate on differences between max SR and max GM portfolios? <br /><br />Thanks<br />P_Serhttps://www.blogger.com/profile/00382284297691291199noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-70161071650766414082018-05-14T08:00:23.076+01:002018-05-14T08:00:23.076+01:00Well it completely changes what you're doing -...Well it completely changes what you're doing - it won't be as diversifying to a long only portfolio. Plus that isn't a statistically significant Sharpe increase.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-79418985640427307822018-05-12T20:29:33.953+01:002018-05-12T20:29:33.953+01:00Hi Rob,
In playing with pysystemtrade, I wanted to...Hi Rob,<br />In playing with pysystemtrade, I wanted to see what happened if I removed the possibility of any short positions. I have heard that long positions usually are more effective than short positions when it comes to trend following. So, I basically added a line to your forecasting file that brought the forecast to 0 if it was negative. The result was a huge bump in sharp from .82 to .95. Do you think this is a reasonable tweak to be made to the system?NewbieTraderhttps://www.blogger.com/profile/04354202871104323102noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-40381515747613334262018-05-10T21:08:16.162+01:002018-05-10T21:08:16.162+01:00Hello all,
While you guys are using native ibapi ...Hello all,<br /><br />While you guys are using native ibapi for placing order (entry/exit), what kind of tool/lib/platform are you using for algo backtesting in python? Can you share some with me?<br /><br />Thanks a lotUnknownhttps://www.blogger.com/profile/06878938966205773451noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-7479780635933868702018-05-10T14:34:13.586+01:002018-05-10T14:34:13.586+01:00Increasing the number of MC runs will improve thin...Increasing the number of MC runs will improve things, but asymptotically (decreasing returns to more runs). I'm not sure whether taking an average of an average is better... I think in the limit and on average the average of X runs of N length should have the same properties as a run of X*N.<br /><br />So if you can afford to computationally then yes, just increase the number of MC runs.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-40130489470214733092018-05-10T13:05:04.509+01:002018-05-10T13:05:04.509+01:00I'm currently re-estimating the weights every ...I'm currently re-estimating the weights every 21 days using bootstrapping. <br /><br />I'm taking gross returns that are all normalised/scaled to a min/max of +/-20. Then for each day I take all the historic returns data available up to that date across all (17) assets and from them randomly sample and combine 20 x 256 day blocks to create one long block. I then make 200 MC runs across that long block and average the samples (256 days each) to get the weights.<br /><br />When doing this using 200 MC runs, I noticed some fairly big variations between re-estimations. A weight might be 0.15 on one occasion but then 0.5 on the next. I realise this isn't necessarily 'wrong' (returns change) but as an experiment I tried upping the MC runs to 1000 and found that the largest change for individual weights between re-estimations dropped to max of about 0.1 (e.g. 0.15 might become 0.25). (I tried various other numbers of MC runs and found that 1000 was about as 'good' as it got - negligible further change reduction above 1000 and pretty much a linear increase below that down to 200 MC iterations.)<br /><br />This prompted me to wonder if there was a conceptually 'better' way to do this:<br />1. Simply increase the MC runs per estimation to 1000<br />2. Calculate an average of the average weights from each estimation (200 runs) either cumulatively or on a rolling window basis and use that average of averages for the weights<br />3. Both of the above <br /><br />Would much value your view on this (even if you think it is just a futile exercise in false precision). Many thanks.<br /><br />Note: I'm using C not Python (so extra MC cycles aren't very computationally expernsive) and the weights are based on the same 6 sets of EMA lengths in your book, but I'm applying those lengths to low pass filters instead of EMAs (because I noticed that the correlation between LP filters of the same periods as the EMAs was considerably lower) <br />AndyWhttps://www.blogger.com/profile/16298595561202011089noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-80478042714204084522018-05-08T09:35:25.233+01:002018-05-08T09:35:25.233+01:00Done. Thanks.Done. Thanks.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-69665122038481891442018-05-05T20:35:23.271+01:002018-05-05T20:35:23.271+01:00Hi Rob, thanks for the great articles. As some fol...Hi Rob, thanks for the great articles. As some folks have already alluded to, it looks like IB has updated their API and the orderStatus() method now takes an extra argument mktCapPrice (http://interactivebrokers.github.io/tws-api/interfaceIBApi_1_1EWrapper.html#a17f2a02d6449710b6394d0266a353313) this is what is tripping up the code in your gist. It would be good to update the gist if you care to keep these tutorials evergreen. Thanks again.Unknownhttps://www.blogger.com/profile/12437513415846850073noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-90047147240587322022018-05-01T21:49:01.392+01:002018-05-01T21:49:01.392+01:00Thanks, that makes perfect sense. I suppose I shou...Thanks, that makes perfect sense. I suppose I should at least simulate live forecasts so I can test the 0 mean hypothesis in a few years from now. Patrickhttps://www.blogger.com/profile/01034323689613977187noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-9281440601705502652018-05-01T16:20:00.264+01:002018-05-01T16:20:00.264+01:00Yes you're on the right track. In fact you cou...Yes you're on the right track. In fact you could break down your execution cost to:<br /><br />A- "delay" - difference between mid close, and the mid price when you actually come to execute<br />B- "bid/ask spread" - difference between mid when you start to execute, and best bid or offer <br />C- "slippage" - difference between best bid or offer, and fill price<br /><br />And I'd expect A to have mean zero, unless you're trading relatively quickly. It's also possible to simulate the effect of a one day delay in your orders (by using the closing price on the following day- pysystemtrade does this by default) and checking to see if you get any improvement in live trading from trading before the close.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-67225863884877136592018-05-01T16:09:03.855+01:002018-05-01T16:09:03.855+01:00Quick follow up. I presume a simpler approach woul...Quick follow up. I presume a simpler approach would be to generate trades based on the closing price, in the traditional way, and simply treat the difference between the closing price and the live price when the execution is done as execution slippage which can be thought of as random noise? I suppose if you take into account any effort put into signal smoothing and possibly adding position inertia/buffering, which introduce meaningful lags into the system, then worrying about a few hours between the forecast generation and trade execution shouldn't actually cause too much concern. Am I broadly on the right track?Patrickhttps://www.blogger.com/profile/01034323689613977187noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-38113521176416896942018-05-01T08:36:46.824+01:002018-05-01T08:36:46.824+01:00You could do that but it wouldn't be correlati...You could do that but it wouldn't be correlation, it would be https://en.wikipedia.org/wiki/Tail_dependence you are measuring.<br /><br />Personally I prefer to measure correlation, and then I impose a risk overlay which measures risk assuming all correlations go to 1,-1 (I'll be blogging about it soon)Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-6176303110591762242018-05-01T00:00:51.973+01:002018-05-01T00:00:51.973+01:00Thanks Rob - appreciate the feedback as always.
Thanks Rob - appreciate the feedback as always.<br />Patrickhttps://www.blogger.com/profile/01034323689613977187noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-32135840427347061902018-04-30T17:29:23.404+01:002018-04-30T17:29:23.404+01:00Hi Rob,
Pysystemtrade calculates instrument correl...Hi Rob,<br />Pysystemtrade calculates instrument correlations based on the past N days of returns. Might it make more sense to base the correlations on the most recent high volatility day (eg., early Feb 2018), since it's at those points in time that diversification really matters? Also, it's at those times when every instrument seems to have a correlation of close to either +1 or -1 to each other. Thanks.NewbieTraderhttps://www.blogger.com/profile/04354202871104323102noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-9049892804068348082018-04-30T09:40:53.181+01:002018-04-30T09:40:53.181+01:00"do you treat this intraday price like just a..."do you treat this intraday price like just another closing price?"<br /><br />Yes<br /><br />To reiterate what I've said before I wouldn't replicate this approach which I've concluded is overly complex.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-1151775536882937432018-04-29T21:23:40.741+01:002018-04-29T21:23:40.741+01:00Hi Rob, I hope all is well. I have read and reread...Hi Rob, I hope all is well. I have read and reread this excellent post. My question is when you append an instrument's intraday price to its daily close timeseries to generate forecasts, do you treat this intraday price like just another closing price? Or do you resample everything to your sampling frequency before calculating ewma values. I see a possible issue with the latter approach as the generic 2300 timestamp will not generate the correct timedelta between closing and sampling. Or maybe its neither of above and some other proprietary technique which I appreciate you would not be able to share? Patrickhttps://www.blogger.com/profile/01034323689613977187noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-6964878126967445232018-04-26T22:34:56.505+01:002018-04-26T22:34:56.505+01:00Thanks Rob.That's a great solution.Thanks Rob.That's a great solution.Gu langyuhttps://www.blogger.com/profile/16054821256101608361noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-17098425418145501622018-04-24T09:09:08.241+01:002018-04-24T09:09:08.241+01:00This seasonality occurs in many places- bonds, com...This seasonality occurs in many places- bonds, commodities...<br /><br />There is no perfect solution, but my own answer is to use a smoothed version of carry by taking a moving average of the raw signal. A lookback of 90 days will take care of quarterly seasonality. A lookback of 365 days will do the same for annual. Both also reduce the responsiveness of the carry signal, so in practice I use a mixture of different smooths.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-45105388264882946242018-04-23T22:01:37.928+01:002018-04-23T22:01:37.928+01:00Hi Rob, thank you for your amazing blog and I am g...Hi Rob, thank you for your amazing blog and I am grateful for your prompt reply to everyone. <br /><br />I have one question regarding carry signal which puzzles me for a while. Hope you could shed some light on it. <br />I discovered that there is "seasonality" in the carry signal of equity indices futures because of the dividend payout period. For instance, to trade CAC40 futures and trade quarterly contracts only in this case. The spread between Generic 1 and Generic 2 futures series during Q1 is always higher than during other quarters (a significant level shift). Just to be more precise, the carry signals calculated using June - March Contracts is higher than the signals calculated using other quarterly contracts. This causes a jump in carry signals at the beginning of Q1 and a gap down at the end of Q1. How do you deal with the "seasonality" issue? or it is OK to leave carry signals being calculated as simple as the spread?<br /><br />Much appreciate if you could help answer this!Gu langyuhttps://www.blogger.com/profile/16054821256101608361noreply@blogger.com