Backtest overfitting
Backtest overfitting is what happens when a trading strategy is selected by searching over its own configurations against the same history used to score it. The winning configuration fits features of that particular sample rather than any structure that persists, so its backtest looks strong and its live performance does not follow.
It is distinct from the failure backtesting treats as cardinal. Lookahead bias is a bug — the simulation let the strategy see the future, and fixing the code fixes it. Overfitting survives a perfectly correct backtest, because the defect is in how the strategy was chosen, not how it was simulated. An event-driven architecture, backtest/live parity, and an exact fill model all leave it untouched.
The correction, held since 2026-08-08
deflated-sharpe-ratio is the published treatment: deflate the observed Sharpe by the expected maximum under the null, given the number of trials and the shape of the return distribution. The consequence for this wiki is procedural — N is an input, so a backtest that does not report how many configurations were searched cannot be corrected by anyone, ever.
The quantity that makes it measurable
pseudo-mathematics-financial-charlatanism supplies what the rest of this corpus discusses only qualitatively: the number of trials, N. Searching N independent configurations of a strategy with no real edge produces a best-of-N in-sample Sharpe ratio that rises with N — seven configurations reach an expected Sharpe of 1 on two years of data, forty-five reach it on five years, in both cases with a true out-of-sample Sharpe of zero. Inverting that gives a Minimum Backtest Length: the history a given N requires before its best result stops being an artifact of the search.
A reported Sharpe ratio is therefore uninterpretable without N, because the same number means "edge" at N=1 and "nothing" at N=45. A backtest that does not disclose how many configurations were tried cannot be assessed for overfitting at all.
Where the corpus generates N without reporting it
strategy-optimization is the mechanism: a hyperparameter search — TPE, Bayesian, CMA-ES-style —
runs the whole backtest hundreds of times and returns the best. That count is N, and no engine in
this spoke reports it alongside the performance it publishes. The same applies to the strategy tables
in awesome-systematic-trading, where each entry's Sharpe comes from its source paper's own backtest.
The strategy tables in awesome-systematic-trading were the same until mid-2026. Re-checked
2026-09-27, they now state the selection: the 61 shown are the strongest of 1,687 replications that
clear t = 1.96, out of 4,843 papers the vendor says it re-ran. That is an N, printed next to the
winners, which is the habit this page asks for, though it is the vendor's own count.
chan-algorithmic-trading names the qualitative version — data-snooping bias, the strategy selected by searching the history it is scored on — as one of seven failures that survive correct code. Overfitting is that entry with a number attached.
The construction that sidesteps it (2026-09-09)
Cover's universal portfolio is the corpus's first result that cannot be overfitted, and it is worth being exact about why. Its guarantee is a worst-case bound over all price sequences, proved rather than measured, so there is no particular history it was fitted to and no N to report — a strategy that runs the entire parameter space at once, weighted by realized wealth, never makes the selection that generates N in the first place (universal-portfolio).
The paper is not an exception to this page, though. Its §8 examples pick two stocks "chosen for their volatility" from a window ending five years before publication, and give no count of how many pairs were looked at. The theorem carries the guarantee; the demonstration is a selected pair, and the two should not be quoted as if they were the same evidence.
Related
pseudo-mathematics-financial-charlatanism · universal-portfolio · backtesting · strategy-optimization · chan-algorithmic-trading · awesome-systematic-trading · beebots · trading-bot-platform