tag:blogger.com,1999:blog-261139923818144971.post4726202626465197773..comments2019-12-09T06:47:03.337+00:00Comments on This Blog is Systematic.: A story of poor statistical intuitionRob Carverhttp://www.blogger.com/profile/10175885372013572770noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-261139923818144971.post-80235938018953191872016-08-31T11:59:06.346+01:002016-08-31T11:59:06.346+01:00Yes this approach is fine. Only 200 points is okay...Yes this approach is fine. Only 200 points is okay, that's the whole point of bootstrapping to get more value out of limited data. <br /><br />The main pitfall is that you will lose any time series dependence (autocorrelation of returns), which you can avoid by block sampling consecutive series of N returns with an appropriate length. This will reduce the effective number of data points you have, so don't make N too big. Maybe N=26 for weekly is okay? (6 months)<br /><br />The alternative is to generate random data with the properties you want as here: http://qoppac.blogspot.co.uk/2015/11/using-random-data.htmlRob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-17579949196553366652016-08-27T21:21:47.804+01:002016-08-27T21:21:47.804+01:00Rob, thanks for these posts, very informative.
I&...Rob, thanks for these posts, very informative.<br /><br />I'm in the process of backtesting a strategy and have a distribution of returns.<br />I'd like to get some statistics on drawdown occurring since the strategy inception, with a view to being able to something like there is 10% probability of drawing down 20% in the first month.<br /><br />As i have this return series, how much sense does it make to create multiple returns series from this distribution via bootstrapping and look at the statistics over these sampled distributions? Is this a common technique? are there any pitfalls to look out for? <br /><br />Unfortunately there is another consideration in that this is a weekly strategy (out of position each week) and i only have aprox. 200 returns.<br />steven joehttps://www.blogger.com/profile/08694296604313395039noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-7384405641653616542016-04-10T13:40:18.136+01:002016-04-10T13:40:18.136+01:00ThanksThanksArman Miahhttps://www.blogger.com/profile/01023780064962157121noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-73900462576965510092015-09-27T05:15:08.471+01:002015-09-27T05:15:08.471+01:00Hi Rob. I didn't realize that you were using t...Hi Rob. I didn't realize that you were using that standard deviation. Thanks for the clarification. <br /><br />One way to avoid the gaussian assumption would be through bootstrapping. I don't know if you want to be bothered but it's just a heads up that there is a way around it. Even bootstrapping probably has some requirements about the "size of the population" you're sampling from so I'm not sure if the fact that you have only 10 observations would effect the boostrapping results. I'd have to look at Efron's bible and I don't have the time at the moment. Still, a nice article that shows how careful one has to be when reading statistical results.<br /><br /> Mark<br /><br /><br /><br /><br /><br /><br />mark leedshttps://www.blogger.com/profile/13213841692738932471noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-15130880580752858762015-09-26T14:00:04.399+01:002015-09-26T14:00:04.399+01:00I'm actually using the conditional standard de...I'm actually using the conditional standard deviation; which I'm taking by using a population estimate based on the 10 known draws. I just quoted the unconditional out of interest. Sorry I clearly didn't make that clear enough.<br /><br />Yes Gaussian is an approximation. But with only 10 observations from the tail it's really hard to think how we could get any meaningful estimate of skew and kurtosis of the tail distribution. <br /><br />I think the fact that the conditional std. dev. in the tails are higher than the unconditional indicates that the overall distribution is non Gaussian ("like, duh" as Bart Simpson might say); assuming the tails are drawn from a different distribution to the body with a higher std. dev. is probably going to give you similar results to modelling the whole distribution using a non gaussian, or a non parameteric approach.<br /><br />Of course in this case we don't have access to the whole distribution, so it's a moot point. Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-68232987998280005112015-09-26T13:10:00.837+01:002015-09-26T13:10:00.837+01:00Rob: I like the approach ( of course, assuming gau...Rob: I like the approach ( of course, assuming gaussian might be questionable ) but when you use the standard deviations in your simulation, I think you're using the unconditional ones when you should be using the sample standard deviation conditional on the other thing occurring ( which is going to be very difficult to estimate ). In other words, I think you should be using the standard deviations of the joint distribution because you'r standard deviation estimate does not take into account that the other event occurred.<br /><br />Definitely what you're doing is tricky so I may be misunderstanding. Thanks for theinteresting post. It definitely got me thinking more about Tom Sawyer's comment about statistics. I forget it exactly but it's something to the effect that statistics can lie.<br />mark leedshttps://www.blogger.com/profile/13213841692738932471noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-38088886120511816472015-09-26T09:42:56.123+01:002015-09-26T09:42:56.123+01:00I have to say I agree with you. I wouldn't run...I have to say I agree with you. I wouldn't run anything I didn't believe in no matter how significant, and I certainly don't do this level of testing for my own stuff. <br /><br />However I think when you're putting something out there for other people to read, you have a bit more responsibility to make sure they interpret it properly.Rob Carverhttps://www.blogger.com/profile/10175885372013572770noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-44078388317242356212015-09-25T22:48:01.127+01:002015-09-25T22:48:01.127+01:00Heh. I find it funny that just as I conclude a ser...Heh. I find it funny that just as I conclude a series of posts about validating every step of a trading strategy development with statistical testing, you come out with a post on it as well. Funny, that.Ilya Kipnishttps://www.blogger.com/profile/11131848727589378549noreply@blogger.comtag:blogger.com,1999:blog-261139923818144971.post-26919439825372504532015-09-25T22:25:24.431+01:002015-09-25T22:25:24.431+01:00I've never been a fan of formal significance t...I've never been a fan of formal significance testing for validating trading strategies or similar. Any attempts to test for statistical significance are flawed because if you look closely enough every assumption they require will be violated in some way (stationary, normally distributed, independent, etc...).<br /><br />At some point you have to sit down and just ask yourself whether you truly believe your findings will persist into the future.Ryanhttps://www.blogger.com/profile/04314718042663866972noreply@blogger.com