Build1 publisher3 min readPublished
Declaring a 0.7 pass mark before the run turned a 0.59 Sharpe into a documented miss
A dev.to trend-following writeup grades every test against thresholds entered in a journal beforehand, and those entries caught an asymmetric benchmark cut and a mislabelled column before the crypto prediction failed.
The Engineer · Build desk

What happened
- A journal entry written before the test predicted that six more Russian futures families would not change the picture and would only add diversification of the same shape.
- Nine families instead of three took the portfolio from Sharpe 0.56 at t 1.5 to 0.83 at t 2.3 and cut maximum drawdown from 19% to 12%, and the six unseen families alone gave 0.88 at t 2.4.
- A cost stress run put the ensemble at Sharpe 1.28 where the baseline had said 0.88, and the improvement under worse assumptions turned out to be a defect in the script.
- The pre-registered crypto test asked for at least 0.7 on the sixteen Binance perpetuals and at least 0.9 on the top four by liquidity, and returned 0.59 and 0.91.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision 0.59 sits above the 0.5 write-off line the entry named, so the single window survives as a candidate while its headline claim goes into the record as failed.
- capability A baseline written down first makes an unexpectedly good result a harness check before it is a finding. Without it the mislabelled column would have been published as an improvement.
- precedent Any project that strips top days from the strategy and leaves the benchmark intact will keep rejecting rules for a concentration its own baseline has more of.
The entry that set those two crypto thresholds also set a floor. Below 0.5, the single 30-day window was to be written off as fitted to one market, and the pre-registration charged 7.5 basis points per side with funding paid [20][21]. The headline result, 0.59 on the sixteen perpetuals, missed its threshold by 0.11 and stayed above the write-off line [23][21]. Restricting the comparison to the same days as the benchmark gave 0.55 and 0.73 [22]. The account stops before it grades the third condition, beating the three-window ensemble on the same samples [29].
Earlier the same day, two checks failed instead of the rule. The project decomposes every positive result by its top five days, on the grounds that an edge which is really three good days is exposure [7]. The daily trend had 49% of its total in five days, all of them in early 2022, and that had been filed against it [8]. Five days out of 1,741 is 0.29% of the sample, and removing them moved the benchmark by 25.4 percentage points [24]. On the same cut the rule was less concentrated than buy-and-hold [10].
The second failure was in the harness. "A number that improves when you make the world worse is not a discovery, it is a bug," the author wrote [30]. The ensemble column carried a hardcoded label while the ensemble the script computed used whatever windows the --lookbacks flag was given; six windows had gone in and the output was read as three [13]. The fix was one line [13].
With the corrections in, the Russian futures picture held across the cuts available. Every window from 30 to 200 days was positive and only the annual windows died [14]. Costs had to move a long way to matter: the real range given is 0.3 to 1.0 basis points per side, times about fifty before the rule falls to the level of just being long [15]. Fifty times that range is 15 to 50 basis points per side [25]. The best single window, 30 days, gave Sharpe 1.29, and 1.34 at t 3.5 with February and March 2022 removed entirely [16]. After the earlier round the rule had been written up as "a positive-expectancy crash hedge, not income" [3]; taking the crash months out made it better [16].
t 3.5 clears the bar set at the start of the project, and nothing else in four months had cleared it [18]. The 30-day window came out of a grid scan run after the fact, and the six new families took part in that scan, so they are no longer clean for this particular claim [19]. For the futures figures to carry to anyone else's book you would need the same instruments, the same sample including early 2022, and the rest of the declared spec: equal volatility contribution per instrument, a leverage cap, weekly rebalancing or a rebalance when the sign flips, costs charged per side, funding paid on perpetuals [1][8]. Code and data are published as trend_daily.py and a repository named strategy-graveyard, which is at least honestly named [28]. Removing the 30-day window's own top 20 days left it positive while the benchmark was deeply negative [17].
What to watch
- Whether the writeup later grades the third pre-registered condition, N=30 against the 60/120/250 ensemble on the same crypto samples.
- A pre-registered N=30 run on a market that took no part in the grid scan, the only test that can clean up the t 3.5 claim.
- Whether the strategy-graveyard repository records the missed 0.7 threshold alongside the passes.