The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality

David H. Bailey and Marcos López de Prado, Journal of Portfolio Management 40(5), 94–107, 2014. The correction this wiki has been naming without holding — marcos-lopez-de-prado already pointed at it as “the companion correction,” and backtest-overfitting describes the disease it treats.

What it does

A Sharpe ratio reported after searching many configurations is not the Sharpe ratio of a strategy. It is the maximum of a sample of Sharpe ratios, and the maximum of a sample is biased upward by construction — the more configurations you searched, the higher the best one looks with no skill involved. The Deflated Sharpe Ratio deflates the observed figure by what the expected maximum would have been under the null, given how many trials were run and how the returns are distributed.

It corrects two sources of inflation at once:

• selection bias under multiple testing — the trial count;
• non-normal returns — skew and fat tails, which the ordinary Sharpe ratio assumes away and which trading returns rarely satisfy.

Sample length enters too: a short track record supports less confidence, and the correction says how much less.

Why it closes this spoke’s edge, and how uncomfortably

The edge asked for a source that reports its N — a backtest or engine that discloses how many configurations were searched. Every strategy and engine in this corpus states a result and not a trial count.

This paper is the reason that omission matters rather than an instance of fixing it. Its argument is that the trial count is an input to the statistic: without N you cannot compute a deflated Sharpe ratio at all, so a backtest published without it is not merely under-documented, it is uninterpretable. The authors are blunt about the scale of the searching — analysts backtest “millions, if not billions” of alternatives.

So the corpus now holds the method for reading a reported Sharpe honestly, and still holds no source that supplies the number the method needs. That is a sharper statement of the gap than the edge had, and the successor edge is written accordingly.

Related

backtest-overfitting · backtesting · marcos-lopez-de-prado · vectorized-backtesting · synthesis