Anyone who has spent more than 5 nano seconds following me on social media or listening to me waffle on during podcasts will know there are a few things that I am not a fan of. One is crypto. The other is AI.
These two things have at least one thing in common. They are terrible for the environment, increasing the usage of energy at a time when we aren't using 100% renewable energy, and using other resources as well.
Both Donald Trump and Elon Musk also like them, and they also sit firmly inside my personal 'not a fan of' box.
And yet, in my fifth book that is being released very shortly, that word 'crypto' is on the cover. Why? Let me explain myself in my own words from the book:
"I should state my own prejudices up front: I am not a fan of cryptocurrencies, which are used primarily for gambling and criminal activities such as money laundering, and which consume a great deal of electrical power, with obvious implications for the climate crisis. In an ideal world I would ignore them completely. Although I am retained as a research advisor for a crypto hedge fund, and I trade Bitcoin and Ethereum futures, I do not hold any crypto in my personal portfolio.
But the point of this book is to guide traders into using certain scientific rules to reduce their chances of losing money, and nobody needs those rules more than those trading crypto. The message of this book as it pertains to crypto can be summarised as “I would not do this stupid thing if I were you, but if you insist, this is the least stupid way to do it.”
In a similar vein, I am not a fan of AI, although unlike crypto I will admit to using it in a very limited way. Mostly for LOLs and making silly pictures, but yes ocassionally for more serious purposes. But I do appreciate that there are a lot of other people out there who are using AI; and perhaps more to the point unlike crypto I do believe AI is useful although it is clearly IMHO massively overhyped in terms of what it can achieve in the quant finance space specifically. But there are an awful lot of people using it in very stupid ways. Hence the subtitle of this post.
A silly picture - thanks chatgpt. See Appendix for prompts and transcript.
The 1st past of this post will be a brief and breezey criticism of the current use cases of AI by people who do similar things to me (develop trading strategies that can then be automated). Then I'll briefly explain the limited set of things I would trust AI to do. Finally I will outline a framework for using AI in a systematic trading strategy reserach process.
Some bad ways to use AI
Let's begin with the dumbest way to use AI for trading purposes, with a prompt like "Tell me what stocks to buy" on a generic public model. I hope I don't need to elucidate to the wise and educated people on this blog why that would be a bad idea.
One level about that would be using the same prompt on a model that has been locally trained on some financial data. Generic machine learning classification tends to do very badly on financial data, mainly due to non stationarity. It's not clear to me why LLMs should do any better.
Going up another level, would be a prompt like "Tell me what trading strategy to use, and why". Congratulations, you have just invented in sample fitting but in a completely opaque way! Plus you can't trust the code that's been used to produce the back test you have just been shown. Interestingly, AIs don't seem to be as good at the insample research loop (develop model -> test-> change model -> retest) that humans do so well.
That brings me to the writing of code. Any code that you use to handle your money and/or other people's money should be 100% trustworthy and fully understood by you, whether it be backtesting or production code. You can never say that is true of AI code.
Agentic AI? Well..
(The second image is a real screen shot from someone I know, who - for obvious reasons - will remain anonymous)
I am expecting this part of the post to get a lot of replies, mostly critical and starting with "Ok boomer..." (typical AI mistake, I am gen-X). I will ignore ones that say "Ah but that is because you should be using model XYZ" or "No you should be using a different prompt" or "Ah but you just need to do X, Y and Z". LLMs have inherent flaws that can never be completely fixed by prompting or using different models or using an agentic framework; and will always be opaque - a very bad thing indeed.
Some good ways to use AI
Idea generation: I am not averse to using AI for idea generation. This could work with eithier a publically trained AI or a local model.
Summarising research: Next after idea generation, it makes a lot of sense if you are doing a lot of research to have an AI summarise for you all the academic papers in field X (you shouldn't ask it to sumarise book Y, since the contents of that book will have been stolen from it's hard working authors). Of course AI's writing style is bloody irritating so I'm not sure how much of it you could read without screaming. I find that asking it do summarys in bullet points removes a lot of the pain. The other obvious problem is hallucinating, so you need to be prepared to get your metaphorical hands dirty and read the underyling research yourself. Still it's a good starting point, and better than just typing "phrase" into google scholar.
Finding data sources: arguably this is something a search engine could do, but we're pretty much at the point where AI is better at search than a search engine is. But don't just say 'give me the data', find out where the date comes from and get it yourself.
Writing small amounts of code: I've used AI to write small bits of code. Small bits because you can easily verify they are correct. It's particularly useful in a situation where the code is relatively isolated from everything else, and it's not a big deal if it fails. A recent example would be that for the framework described at the end of the post, I wanted a configuration file that could only be added to and not otherwise edited, and decided to implement this using hashes (yeah I know that makes the file like a single node blockchain, the irony isn't lost on me).
I could have easily written this myself in an hour or so, but I was in a hurry and got AI to write it. It ran perfectly and although the style of the code wasn't something I would have written myself, it's small enough that I can easily check it's correctness in much less than an hour (so net time saved), and if it goes wrong it isn't going to result in me accidentally buying 10,000 contracts of Bund futures.
Checking code: In theory at least, I don't have a big problem with AI checking your code and suggesting changes. I've had other people check my code with AI, and to be fairness it did find some issues. The downsides here is that the language of code checking in AI is bloody awful (as is obvious from the AI contributions to pysystemtrade), and you often need another AI to interpret what it is saying, which seems to be a net waste of time.
You should absolutely not merge the changes from the AI without checking them. The same code check someone did for me resulted in the AI unintentionally introducing an absolutely massive bug. So what I am suggesting here is definitely not a 'write code with AI', 'check code with AI' doom loop without human intervention; or using AI so extensively that you can't actually properly check all the code written or suggested changes. That would be insane, and it's incredible to me that there are people doing it.
Writing tests: I hate writing tests, as anyone involved in my open source psystemtrade project will know. Tests are small bits of code with a well defined objective that if they go wrong won't affect the rest of your codebase - perfect for AI. Note that I wouldn't extend this to writing documentation, another thing I am really lazy at (although unlike tests, I do actually do it), since AI writing isn't something I am a fan of; and there is a real benefit to writing your own in my opinion.
Stealing code: Like crypto, AI is also good for doing crime and stuff.
A framework for using AI (and also human brains)
With the above in mind, what is a safe framework for using AI in trading research? In fact this is exactly the same framework that should be used in an ideal environment for testing human generated ideas. We still have the same three versions of overfitting we're trying to avoid: explicit, implicit and tacit. It's just that the overfitting is wrapped in an opaque blob so we don't even know if it's happening or not.
Idea generation, data scoping, initial research: As noted above this could be done by AI, or by a person. It's fine.
The end point of this exercise will be the following:
- Some trustworthy data. To be trustworthy the data must have come direct from the source not via the AI. You should have checked the specs of the data to make sure there is no in-sample fitting hidden inside it (this is a problem often for sell-side produced 'strength indices', and more recently I've had this issue with certain Glassnode crypto data) and no forward looking data (a problem for eg macro data published with a lag and then revised, and company earnings that certainly don't come on the final day of the quarter they relate to).
- A piece of parameter-isable code implementing the strategy idea (i.e. producing a forecast) given the data as an input and incorporating parameters that require calibration like the fast_span and slow_span below:
(As an aside, the code produced here was a bit rubbish as it also included some backtesting fluff and didn't just produce the forecast but this is a 'skill issue' that better prompting or a skill.md file can probably solve).
Note that the parameter-isable code is small and therefore it's correctness can be verified. However it may well incorporate tacit fitting.
That is the last time we will touch AI in this process. Everything else that happens will happen in a robust testing framework. Importantly that robust framework must not have been written with AI or be run be agents (with the caveat that small parts of the code may be AI written, AI may have been used to suggest improvements, and the tests may be written with AI).
A good way of explaining this is to think about the harness+AI that produces AI agents, something like this:
Instead of a model inside a harness (and inside a sandbox here) the model only gets to play with one thing; the parameter-isable python code that ends up getting tested as a forecasting rule. Everything else in the robust framework around it is off limits to the AI.
The list of requirements for the robust tester is again not different from what a proper tester would have for human generated ideas or strategies:
- Generating the parameter fitting space eithier by using a very limited amount of real "burn in" data, fake data or theory. If real data is used, performance isn't considered, only correlation and turnover.
- As alluded to above when I discussed the config file hash key thing, an add only system where new forecasting rules once tested can't be removed from the backtest. This avoids implicit fitting in the form of 'research loops' where the results of backtests are used to refine forecasting rules, or remove them entirely. As we're not ever going to remove or drop a rule, we don't need to worry about nonsense like Whites Reality Check and we're not at risk of dropping rules that may work but whose natural profitability is too low to pass an arbitrary statistical hurdle.
- Backward looking explicit fitting of the combination of optimal parameters and forecasting rules.
- The fitting is robust - it knows about statistical significance and trading costs (as discussed in my recent series of posts, the last one is here).
- Both positive and negative versions of a strategy to avoid tacit in sample fitting. This avoids the problem that both smart humans and AI already know what's worked, eg medium speed trend following.
What I haven't done of course is specify exactly how this robust fitting engine will work. That is something I am currently working on; and will be spelt out more in (a) future blog post(s). It will also be the subject of my next (6th) book, title to be confirmed, publication date probably at least 2 years away...
Appendix: chat transcript for image
Me: I want a picture of renowned systematic trading author robert carver sat at his computer swearing because AI has just deleted all his files
Chat:
Me: that doesn't look like him and that book cover is completely wrong
Chat: You're right — the likeness and the book cover both need correcting. The image invented a generic older author rather than capturing Robert Carver. Let's get it right this time. Could you upload: 1. A photograph of Robert Carver that you'd like me to use as the likeness reference. 2. The actual cover of his book, Systematic Trading.
I'll recreate the scene with: - A much closer facial resemblance. - The genuine book cover, not an AI-invented version. - Robert visibly furious, shouting and swearing at his computer. - A screen displaying: "AI HAS DELETED ALL YOUR FILES".
Would you prefer photorealistic comedy or an exaggerated satirical caricature?
Me: <Attached Images> I am robert carver and I have the copyright to these images. photorealistic
Chat:
Me: WTF is going on with my hands, they look like those of an 80 year old man? What OS is that - should be linux? And why is my book so big - should be smaller
Much better on the Linux front! This version has: - Ubuntu-style Linux desktop, with Tux visible. - A smaller, more realistic-sized book on the bookshelf. - A ThinkPad rather than a generic desktop monitor. - More natural-looking hands and gestures. - Your actual appearance and clothing as the reference. The AI has still managed to delete everything. At least it hasn't deleted your book. 😄
NOTE: Smiley face was inserted by AI not me. I am not that whimsical










