Backtest Validator — Strategy Production-Readiness Decision Tool

Purpose

Most trading-strategy backtests look great in-sample and disappoint live. The gap comes from a small number of well-understood causes:

1. Selection bias — testing N configs and picking the best inflates Sharpe by ~√(2 ln N). For N=120, that's a 3.1× inflation. The "winner" of a sweep is guaranteed to look amazing — but only because of the luck embedded in trying many things.
2. In-sample-only evaluation — without walk-forward / out-of-sample tests, you don't know if the rules generalize beyond the window they were tuned on.
3. Unrealistic fills — backtests assume you get the stop level exactly. Live brokers don't always cooperate. A strategy that breaks at +1 tick of slippage is a strategy that loses money in production.
4. Regime concentration — profits often live in one favorable quarter. Strip that quarter and the equity curve dies.
5. Tail-event dependency — a few outlier wins masquerade as a robust edge. Strip the top 5% of trades and many "winning strategies" turn negative.
6. Filters doing the wrong work — comparing against a random-entry baseline often shows the entry signal adds zero edge; the exits are doing all the work.

This skill runs all of these checks and produces a decision document the user can act on.

When to invoke this skill

Any time the user is moving from "I have a backtest result" to "should I deploy this?" Specifically: