← Projects·Quantitative Researcher
Multiple Testing & the Deflated Sharpe Ratio
A study in multiple-testing discipline. Sweep 160 BTC trading rules, rank by in-sample Sharpe, then apply the Deflated Sharpe Ratio (Bailey & López de Prado, 2014). The rank-1 rule looks great until you condition on the number of trials.
The hypothesis
If you test 160 coin-toss strategies on a BTC price series, the best one will look like an edge just by luck. Conditioning on the number of trials is what separates a discovered signal from noise that happened to win.
What the project does
- Sweeps 160 parameterizations of a single rule family across BTC.
- Ranks them by in-sample Sharpe — the rank-1 looks like a 1.14 Sharpe.
- Generates a block-bootstrap null with the same autocorrelation as the source.
- Asks: what Sharpe would rank-1 achieve on pure noise?
- Computes the Deflated Sharpe Ratio (DSR) per Bailey & López de Prado (2014).
The result
- In-sample best Sharpe: 1.14
- Pure-noise expectation for rank-1: ≈ 0.92
- DSR: 0.70 — a fail under the standard 0.95 cutoff.
What’s transferable
The discipline generalizes. Every research shop with a parameter grid needs a multiple-testing correction; the 0.95 threshold is the difference between a published edge and a graveyard of overfit losers. The bootstrap-null methodology here runs in 20 lines and ships with the repo.