MEDIOR · TECHNISCHE ANALYSE

Backtesting (Sanity Checks)

Overfitting, sample size, regime changes, and why paper results differ from live.

Bouw een compleet handelsplan met entry, exit en risicobeheer.

What Is Backtesting?

Backtesting is the process of applying a trading strategy to historical data to see how it would have performed. It's the closest thing to a "test drive" for a strategy before risking real capital.

However, backtesting is full of traps. A strategy that looks amazing on historical data can fail completely in live markets. Understanding why is essential.

Overfitting: The #1 Trap

Overfitting occurs when a strategy is tuned so precisely to historical data that it captures noise rather than genuine patterns. An overfitted strategy "memorizes" the past instead of learning generalizable rules.

Signs of overfitting:

  • The strategy has many parameters (5+ adjustable settings).
  • Small changes to parameters dramatically change results.
  • It works perfectly on one specific time period but fails on others.
  • The equity curve looks "too good to be true" - no drawdowns, constant profits.
Rule: If it only works on one month or one market, it probably doesn't work. A robust strategy should show positive results across multiple time periods and (ideally) multiple instruments.

Sample Size Matters

A strategy that produced 5 winning trades is not statistically meaningful. You need enough trades to distinguish skill from luck:

  • Minimum: 30+ trades for basic statistical significance.
  • Better: 100+ trades across different market conditions.
  • Best: Multi-year data covering bull markets, bear markets, and ranges.

A 90% win rate over 10 trades means almost nothing. A 55% win rate over 500 trades with positive expectancy is far more meaningful.

Regime Changes Break Edges

Markets change. A strategy that worked in 2020's volatility may fail in 2023's low-volatility grind. Common regime changes:

  • Volatility regime: Low vol → high vol (or vice versa).
  • Trend regime: Trending → ranging.
  • Correlation regime: Assets that moved independently start moving together (or vice versa).
  • Structural changes: New regulations, market structure changes, new participants.

No strategy works forever. The question is: does it work across enough regimes to be useful?

Transaction Costs and Slippage

Many backtests ignore real-world execution costs:

  • Commissions: Even small fees compound over hundreds of trades.
  • Spread: The bid-ask spread is a cost on every entry and exit.
  • Slippage: In live trading, you rarely get the exact price shown in the backtest.
  • Market impact: Large orders move the market against you.
Reality check: A strategy with a 0.1% edge per trade can easily become unprofitable after accounting for a 0.05% spread + 0.03% commission + 0.05% slippage. Always include realistic costs in your backtest.

In-Sample vs. Out-of-Sample

The gold standard for backtesting is to split your data:

  • In-sample: The data you use to develop and optimize the strategy.
  • Out-of-sample: Data the strategy has never "seen." You test on this data only once, after development is complete.

If the strategy performs well on out-of-sample data, it's more likely to be robust. If it only works on in-sample data, it's probably overfitted.

Three Questions for Any Backtest

Always Ask:

  • 1. Is the sample size large enough? (30+ trades minimum, 100+ preferred)
  • 2. Does it work across different market regimes? (Bull, bear, range, high vol, low vol)
  • 3. Are transaction costs included? (Spread, slippage, commissions)

If the answer to any of these is "no," the backtest results are unreliable.

Key Takeaways

  • Backtesting is essential but full of traps.
  • Overfitting = memorizing noise instead of learning patterns.
  • Small sample sizes can't distinguish skill from luck.
  • Regime changes break edges - no strategy works forever.
  • Always include transaction costs and use out-of-sample testing.

Wat is een handelsplan?

Een handelsplan is een geschreven document dat je regels definieert: wanneer je handelt, hoe je handelt, hoeveel je riskeert en wanneer je stopt.

Componenten van een handelsplan

  • Markt: Welke instrumenten handel je?
  • Tijdsframe: Op welk tijdsframe neem je beslissingen?
  • Entry-criteria: Welke voorwaarden moeten aanwezig zijn om een trade te openen?
  • Exit-criteria: Wanneer neem je winst? Wanneer snij je verlies?
  • Risicobeheer: Hoeveel riskeer je per trade? Per dag?

Belangrijkste punten

  • Een plan voorkomt emotionele beslissingen.
  • Schrijf je plan op VOORDAT je handelt.
  • Evalueer en pas je plan regelmatig aan.
  • Discipline is belangrijker dan de perfecte strategie.

Les afgerond

Je hebt afgerond: Backtesting.