Skip to content

Strategy testing, explained

Can you trust your backtest? Learn how to find out.

Straight answers to the questions traders ask most about testing a strategy: overfitting, luck, costs, how much evidence is enough, and why backtests fail live. Plain English, no hype.

The questions traders ask

Gathered from trading forums, communities and search, grouped by topic. Each has a short answer and a place to go deeper.

Real edge or luck

The question behind every other question: can you trust this result?

Is my strategy a real edge, or just luck and overfitting?

You can’t tell from the backtest alone. It needs to hold up on data it was never tuned on, after you account for how many versions you tried, and with realistic costs included.

Read the guide
When should I stop tweaking my strategy?

Decide what you’ll test before you see results, and stop when a change has no reason behind it other than a better backtest. Once you’ve opened your final test, you’re done tuning.

Guide: when to stop
How do I keep part of my data untouched for a final test?

Set a stretch of history aside before you start, build and tune on the rest, and run the finished strategy on it once. Every extra look turns it into tuning data.

Guide: keep a sealed test

Backtest vs. live

Why a strategy that looked great can disappoint with real money.

Why does it work in the backtest but lose money live?

Usually one of four reasons: it was overfit, the backtest left out real costs and fills, a bug or look-ahead mistake flattered it, or the market it was built on has changed. Sometimes it’s simply a normal losing stretch that the backtest had too.

Myth: live should match
How do I account for slippage, commissions and realistic fills?

Charge every trade the commission and fees you’d really pay, add slippage on entries and exits, and only assume fills that could have happened. Then re-run with higher costs to see if the edge survives.

Glossary: slippage
Could a bug or a peek at the future be making the profits?

It happens more than people think. The classic mistake is look-ahead bias: deciding a trade with information you couldn’t have had yet, like a bar’s close. Results that look too good deserve a line-by-line check.

Glossary: look-ahead bias
Will my strategy still work when the market changes?

No one can promise that. What you can do is test how it did across different conditions, calm and volatile, trending and choppy, so you know where it struggles before the market shows you.

Glossary: market regime

How much evidence is enough

Trades, years and numbers: what actually tells you something.

How many trades do I need before I can trust a backtest?

There’s no magic number. Rules of thumb like 30 or 100 trades aren’t proof. It depends on how big the edge is compared with the noise, how many versions you tried, and whether the trades span different market conditions.

Glossary: sample size
How long should I forward-test before using real money?

Long enough to see the strategy behave the way the backtest said it would, across more than one kind of market. Forward testing is slow at proving an edge, so an honest backtest matters even more.

Glossary: forward testing
Which numbers matter: win rate, drawdown, profit factor, Sharpe?

No single number. Read the win rate together with the average win and loss, look at the worst drawdown and the number of trades, and treat profit factor and Sharpe as summaries of the same trades. Every one of them is easy to inflate by overfitting.

Glossary: expectancy

Tools, code and AI

Testing without a programming degree, and what to ask of any tool.

Can I test a strategy without learning to code?

Yes. No-code backtesters and AI tools can turn written rules into a test. Whatever you use, ask the same three things: can it show you the actual trades, the costs it assumed, and the checks it ran?

Myth: trust the tool
Can AI test my strategy, and can I trust what it gives me?

AI is good at turning a description into rules and suggesting ideas. Trust the numbers only if they come from a real backtest you can inspect, not from text the AI wrote. Ask to see the trades and the checks.

Myth: trust the tool
Is backtesting even worth it?

Yes, mostly for what it rules out. A backtest can’t promise a strategy will work, but it’s the cheapest way to find out that one doesn’t, before real money does.

Myth: backtesting is useless
How do I test an ICT or discretionary strategy?

Turn it into rules first: exactly what has to be true to enter, where the stop goes and when you exit. The parts that stay “it depends” can’t be tested honestly, so make them as specific as you can.

Glossary: discretionary trading

Prop firm challenges

Profit isn’t enough when the rules can end the attempt.

Is my strategy good enough for a prop firm challenge?

A challenge asks whether your strategy can reach a profit target without breaking rules like a daily loss limit or a maximum loss. A strategy that’s profitable overall can still fail on one bad day, so test it against the rules, not just for profit.

Glossary: prop firm challenge