Wordle (TM) and the one simple hack you need to pass funded trader challenges

An unusual (but quick) mid month post, as this is a live issue I thought I'd publish this whilst it's relevant.

There has been some controversy on X/Twitter about 'pay to play' prop shops (see this thread and this one) and in particular Raen Trading. It's fair to say the industry has a bad name, and perhaps this is unfairly tarnishing what may pass for good actors in this space. It's also perhaps fair to say that many of those criticising these firms, including myself, aren't as familiar with that part of the trading industry and our ignorance could be problematic. 

But putting all that aside, a question I thought I would try and answer is this - How hard is it to actually pass one of these challenges? As a side effect, it will also tell us what the optimal vol target is to use if we're taking part in one of these challenges. Hence the clickbait article heading. I know from experience this will open me up to having to filter out 500 spam comments a day, but f*** it. 

As well as modelling Raen, I also model a much dodgier challenge later in the post, from another company which I will name only as prop firm #2. Finally I close with some generic and unquantified thoughts on the subject. 

Standalone Python code here. You can play with this to model another firms challenges.

TLDR: 

• Raen you have reasonable chance of passing their first round challenge and you should use a vol target of [scroll down to find out!] to maximise your chances.
• Prop firm #2 and most of the 'industry' use a very long bargepole, I can lend you mine
• I remain skeptical of pay to play

As to what any of this has to do with the word game Wordle (TM), read on to find out.

IMPORTANT: This is not an endorsement of Raen. I have no association with them and I remain skeptical of this entire industry. Their CEO reached out to me after this blogpost was initially published, confirmed my understanding of the challenge parameters was correct, and gave me permission to use the firms name. I made one small correction to the post as a result of that contact.

The (relatively) good guys 

The rules of the Raen challenge are this:

• You must make 20%
• You can't lose more than 2% in a single day. There is no maximum trailing drawdown. So if you lose 1.99% every day forever, you're still in the game.
• You must trade for at least 30 trading days before passing the challenge
• It costs $300 a month to do the challenge. This isn't exactly the same as the (completely arbitrary and made up) prop firm which charges a little more, but it makes it easier to directly see how many months we expect to take by backing out from the cost per month. I assume this is paid at the start of the month.

Note: this is just the 1st stage of the challenge. The rules for the 2nd stage are much more nebolous, but to be fair there are no charges for those. Like I said, this prop firm appears to be amongst the relatively good guys. 

 

I've also got these parameters:

• 256 business days a year, 22 business days a month (it's actually more like 21, but again this higher figure will make the prop firm look good)
• Random gaussian returns generated with no autocorrelation. This is extremely kind as it ignores the chance of fat tails that are somewhat common in finance.
• If we get stopped out we try again, which means restarting the challenge from scratch. There are no reset fees. I assume that this reset doesn't affect the timing of monthly fees (I can't find the answer to this question on the website). This must be the case as otherwise the cost of resetting would be free and your best strategy would be to keep making massive bets every day and you would pass eventually and only ever have to pay the first month.
• We give up if we can't pass after trying for a year (there are no time limits in the challenge, but this speeds up the computation and seems like reasonable behaviour).
• I assume there are no other limits which make it hard to hit a given risk target. This is unlikely to be a constraint except for suboptimally high vol targets.

There are two clear variables we are missing: the expected Sharpe Ratio, and the vol, both required to generate the gaussian returns. The former is assumed to be exogenous (a question to answer is how hard are these challenges to pass - if you need a SR of 4 to pass them that suggests they are probably too hard), whilst the latter we can optimise for. Note that due to the drawdown and self imposed time limit the optimal vol target won't be equal to the usual Kelly optimal. In fact this subject is intellectually interesting as well as topical since it's the first time I've looked at optimisation with a drawdown/time constraint. 

I run this as a bootstrap exercise. We try and optimise: (a) minimise the median cost, (b) maximise the probability of being funded before we give up. 

OK so two simple graphs then. Each has a different line for each SR, and the x-axis is the vol target we are running at. The y-axis on graph one is the cost, with a minus sign so we have the natural thing of a high y-axis being good. On graph two the y-axis is the probability of passing before we give up, again obviously high y-axis is good.

Median cost of getting to stage two, lines are SR, x axis is annual vol target, y axis is cost (bigger minus numbers are higher costs)

Note that for SR/vol combinations where we have a less than 50% median chance of succeeding the median cost will be equal to the monthly cost * 12. This is the case for SR<1.5

Probability of getting to stage two, lines are SR, x axis is annual vol target, y axis is probability of success

What conclusions can we draw from this?

• The optimal vol target depends on your SR and whether you are focusing on costs or probability*
• To get a greater than 50% chance of passing we need an expected SR of 1.5 or higher. 
• The expected median cost with optimal vol is going to be be $2000 for a SR of 1.5, which you can get down to $1500 if you are the next RenTech (SR of 3). 
• The expected median time to pass is going to be about 7 months for a SR of 1.5 or about 5 months if you are the next RenTech

* Experts will recognise the vol target choice as the Wordle (TM) starting word problem (yes we finally got there). The best starting word for Wordle will depend on whether you are maximising your probability of winning, or trying to minimise the number of guesses you make. Similarly, are we trying to maximise our chance of passing the challenge, or minimising our likely cost? They are not quite the same thing.

The optimal vol looking at costs is around 15 - 20%. Looking at probability of passing, it's around 12% for very high SR traders, and more like 22% for low SR traders. Basically if you're crap you have to take a bit more risk to have a chance. If you're good you can chill. Given we're assuming Gaussian returns I'd be tempted to mark these figures down a bit, although note that for high SR traders using less than optimal vol is quite harmful (very steep lines) whilst using more than optimal is less painful (this is completely at odds with Kelly of course).

Since nobody knows what their SR is, I'd suggest using 15% as a vol target. If you are incredible that is slightly more than optimal, but you still have an 80% chance of passing. If you are less incredible it may be slightly less than optimal, but then you have no business passing this challenge anyway.

The not so good guys firm #2

Here is an example of another firm's level 1 challenge, I won't name them but they are currently on the 1st page of google results for the search term "trading prop challenge" so that narrows it down. This firm has several challenge tiers in the futures space, I've chosen the lowest; but all the conditions are the same just different notional capital and $ costs. 

The rules of the challenge are this:

• You must make 6%
• The maximum drawdown is 4%; trailing based on daily balances.
• If you lose more than 2% in a day, well basically you're stopped out at 2% but the challenge doesn't end. So your max loss in a day is 2%. In practice would be slightly more because of slippage but let's be generous here.
• There is a one time activation of $130 (not exact figures again but ballpark).
• You have to do the challenge in 30 days. It costs $100 to start each challenge. If you want to extend the challenge by 30 days it costs another $100. This equates to a monthly fee of $100, so we'll model it like that.
• If you need to reset (start again because you've gone boom) it's $80. This is on top of the monthly cost since it doesn't reset the number of days to zero before you have to pay a monthly fee again.
• There are optional data fees we will ignore, because there are enough fees here already.

Now, it's worth saying that there are many other terms and conditions that make firm #2 much dodgier and less likely to fund you or give you your profit share once funded (of course we're assuming firm #1 sticks to their word as well); but we're purely here to model the challenge itself.

Here are the graphs:

Median cost of getting to stage two, lines are SR, x axis is annual vol target, y axis is cost (bigger minus numbers are higher costs)

Probability of getting to stage two, lines are SR, x axis is annual vol target, y axis is probability of success

This is not what I had expected. I had expected the challenge to be much harder, so the firm could keep collecting the fees. But this challenge is easy to pass, just use vol more than 25%. Basically you get to flip a coin a couple of times and sooner or later it will turn up heads. This strategy will work even if you are a losing trader (SR -0.5) as shown. The only benefit of being a better trader is you will pass quicker and thus pay less. 

This is an incredibly badly designed challenge. It rewards higher volatility. It doesn't discriminate at all between good and bad traders. 

So eithier (i) there are other conditions in the (very hard to find) small print that in practice make the challenge hard to pass or (ii) it's a deliberate strategy to allow almost anyone to get to the next stage. The biggest red flag is that trading with this particular firm is sim only even after you have passed the challenge. They don't want to make the initial challenge too hard; they want you as a paying customer ASAP. And people who use too much vol are ideal customers for bucket shops.

The prop firms view

Of course what we're not doing here is looking at things from the prop firm's point of view. The challenge is designed to answer the question: "is this potential trader any good or just lucky?". At least that is if you are assuming they are genuinely looking for good traders. Which prop firm #2 definitely isn't, so let's focus on Raen.

The main shortcoming of these challenges is that 30 days or even a year is a wholly insufficient time to determine if anyone has any skill, unless they are very highly skilled indeed. And again, to be fair, the initial challenge of Raen is purely a screening exercise that will essentially tell you (a) if someone has a vague idea of how to manage risk and avoid a 2% daily drawdown and (b) is eithier very good (SR somewhere over 1) or just very lucky.

Someone who shoots for a vol target that is too high will almost certainly fail. However there is still a chance of a crap trader being lucky. But hopefully the second stage will weed them out. So we aren't too worried about type 1 errors.

However even relatively highly skilled traders (say SR 1 to 1.5) will only have a coinflip chance of passing. So there is still quite a big chance of a type 2 error and missing out on the next Nav Sarao*. Perhaps that's okay. They're probably only interested in people with a SR of over 2 anyway, where the passing percentage for a year will be over 60% if they use optimal vol. Of course I'm assuming all these people have several thousand dollars to stump up to a years worth of monthly fees. There will be many who don't, and therefore also miss out on potentially being funded even if they are good traders. So I would say the possibility of a type 2 error is quite high.

* this famous gentlemen who for all his faults was an incredibly succesful futures prop trader even when he wasn't breaking the law.

 

I'd say on balance that Raen's challenge is relatively well designed given all the caveats. It's simple, it's difficulty rating feels about right, and the 2% daily drawdown acts as a simple anti muppet filter. The fact there is an optimal vol is satisfying. It would be interesting to see their internal numbers on how many people pass the first and then the second challenge; and then go on to become good traders. That will tell us what their type 1 error actually is. 

But is this all really a good idea? Some unquantified and unqualified opinions

Putting aside the statistical debate, is this all really a good thing? For the traders trying out, or for the firms themselves (assuming they again are genuine). There are many red flags in this industry, having to pay to be considered for a 'job' is always bad (although Raen's CEO clarified to me that they also accept applicants who haven't passed the challenge, presumably with some kind of filter on experience); the fact that many places are purely bucket shops where you trade against the broker is awful (again not Raen), frankly the whole thing makes my stomach churn but I'm trying to be as fair as possible here and put emotions aside.

The world of trading has changed an awful lot. In this post the founder of Raen says their shop is for people who would never the opportunity to get into Jane Street (JS). But JS is looking for people with a very particular set of skills to do a certain kind of trading which you can't do unless you have the sort of resources JS has. 

Yes we can argue that the Jane Street filter is too strict (though they hired SBF, a man who did not understand how to size trading positions, so maybe not strict enough), but it's pretty silly to pretend that Jane Street would be interested in hiring the sort of people who have the ability to be point and click futures traders. It's really not the same at all.

Raen apparently has ex JS people working there and they are 'very succesful'. I am sure they are. I'm also sure that they're almost certainly not point and click traders eithier. But is it really realistic to replicate JS by hiring a completely different set of people, without any of the filters JS uses to get specific sets of skills, using a totally different process from what JS uses, and without most of JS resources; and then sit them next to ex JS traders from whom they will presumably absorb brilliance by osmosis?

Basically if for some reason you are trying to be the next JS why are you using a hiring process which is clearly for point and click traders? There are no references to APIs that I can see on any of these challenge websites, so I assume it's point and click they are looking for.

So, is the world of point and click prop traders too inaccessible? It's probably more accessible than it was 20 years ago from an IT and cost perspective. But admittedly if you're not trading costly and dodgy retail assets,  and want to trade futures, then no you can't really do this with the $3000 or so you'd need to pass even a good trading challenge. The $100k of (notional, real?) money you get from Raen is the bare minimum I suggest in my book. You would need less in equities though, but to be eg US PDT you need $25k (for now). 

But from my perspective, the whole point and click futures industry seems very... niche. The vast majority of professional traders now are basically quants, or heavily supported by quants, and/or using data other than the charts and order books fancied by the dozen monitor setups of the cliched point and click trader. It's an area of the market that really is very efficient and where the vast majority of point and click humans can't compete even if supported by execution algos, which is why I deliberately trade much... more... slowly. 

In fact I'd say there are now significantly more people employed by the likes of JS than by genuine and profitable point and click firms. 

So we're talking about getting access to a relatively tiny industry that is frankly a bit quaint and probably still shrinking. I can understand why many people want to get into it though. Who wouldn't want to gamble for a living, make millions of dollars a year, in a job which requires no qualifications (no Phd in astrophysics needed here!), which so many films and YouTube videos have glamorised, and which eithier requires almost no work or where hard work and effort will be rewarded (depending on which video you watch). 

I can believe that there are a very small number of people who have pointed and clicked for so long, that they really do have an ability to 'feel' a particular market very well, they can glance at an L2 order book and see patterns, they know which news and statistics to focus on, they know what other markets to look at, they know how to manage risk and size positions, they have built execution algos that enable them to compete not with HFT but certainly in the sub one day area... and they are certainly better traders than me or my systems. 

As to how you would select such people, I do not know. They are not my people. Personally I am very skeptical as to whether there really are people who can sit at a computer having never traded futures before except maybe in a simulator, with no training or market experience, and have some innate trading ability that enables them to have a high probability of passing a trading challenge with the sort of SR that would make most hedge funds weep with envy, and also that those challenges are the best way of being able to tell that someone has that innate ability. 

But once again, I'm not in this industry so what do I know. 

Summary

Raen: not a bad intial screening and a more than 50% chance of a pass with a SR above 1.5. But it will cost you more than $300. Budget for several thousand bucks and use a vol target of around 15% to optimise your chances.

Unamed prop firm #2 I googled and most of this industry: stay away for gods sakes.

Pay to play: morally dubious IMHO

Random futures traders having some sort of innate talent that can be found in this way: I doubt it