Backtest overfitting
Backtest overfitting is what happens when a trading strategy is selected by searching over its own configurations against the same history used to score it. The winning configuration fits features of that particular sample rather than any structure that persists, so its backtest looks strong and its live performance does not follow.
It is distinct from the failure backtesting treats as cardinal. Lookahead bias is a bug — the simulation let the strategy see the future, and fixing the code fixes it. Overfitting survives a perfectly correct backtest, because the defect is in how the strategy was chosen, not how it was simulated. An event-driven architecture, backtest/live parity, and an exact fill model all leave it untouched.
The correction, held since 2026-08-08
deflated-sharpe-ratio is the published treatment: deflate the observed Sharpe by the expected maximum under the null, given the number of trials and the shape of the return distribution. The consequence for this wiki is procedural — N is an input, so a backtest that does not report how many configurations were searched cannot be corrected by anyone, ever.
The quantity that makes it measurable
pseudo-mathematics-financial-charlatanism supplies what the rest of this corpus discusses only qualitatively: the number of trials, N. Searching N independent configurations of a strategy with no real edge produces a best-of-N in-sample Sharpe ratio that rises with N — seven configurations reach an expected Sharpe of 1 on two years of data, forty-five reach it on five years, in both cases with a true out-of-sample Sharpe of zero. Inverting that gives a Minimum Backtest Length: the history a given N requires before its best result stops being an artifact of the search.
A reported Sharpe ratio is therefore uninterpretable without N, because the same number means “edge” at N=1 and “nothing” at N=45. A backtest that does not disclose how many configurations were tried cannot be assessed for overfitting at all.
Where the corpus generates N without reporting it
strategy-optimization is the mechanism: a hyperparameter search — TPE, Bayesian, CMA-ES-style — runs the whole backtest hundreds of times and returns the best. That count is N, and no engine in this spoke reports it alongside the performance it publishes. The same applies to the strategy tables in awesome-systematic-trading, where each entry’s Sharpe comes from its source paper’s own backtest.
chan-algorithmic-trading names the qualitative version — data-snooping bias, the strategy selected by searching the history it is scored on — as one of seven failures that survive correct code. Overfitting is that entry with a number attached.
Related
pseudo-mathematics-financial-charlatanism · backtesting · strategy-optimization · chan-algorithmic-trading · awesome-systematic-trading