How to backtest a trading strategy: 8 steps and a sample test plan
Write the rules, split the data and set a pass bar before the first run; then charge real costs, log every version, and open the sealed block once. That order keeps a backtest from grading your own tuning.
Key takeaways
- Backtesting a trading strategy means running exact rules over historical data with realistic costs, then judging the result on data you set aside first.
- Split the history into build, check and sealed blocks in date order, and write down what counts as a pass before the first run.
- Log every version you try. Three stops, three targets and five time windows already make 45; grid versions overlap, but Bailey and colleagues show that on five years of data, 45 independent tries can make one look strong by chance.
- Passing the check and sealed blocks is a reason to forward test; live results usually come in weaker than the backtest.
To backtest a trading strategy, write its rules so precisely that anyone would take the same trades, run them over historical data with realistic costs, and judge the result on data you set aside before you started. The order is what makes it honest. Decide how you’ll judge the test before you see a single result, or the backtest ends up measuring your tuning instead of your strategy.
How do you backtest a trading strategy, step by step?
Backtesting a trading strategy takes eight steps: write exact rules, split the data by date, set a pass bar, set realistic costs and fills, run and log every version on the build block, read the trades, run the check block once, and open the sealed block last. The first three happen before you run anything.
- Write the rules so a stranger would take the same trades. Spell out the entry, the stop, the exit, the position size, the hours it trades and what happens at the session close. Include the awkward cases, such as a second signal on the same day. If two people could read your rules and take different trades, a judgment call is still hiding in them. (Not sure whether to test by hand, in a spreadsheet or in code? See the comparison of methods.)
- Split the history into three blocks, in date order, before you look. The first block is for building and tuning, the second for checking the finished version, and the last stays sealed until the very end. Never shuffle bars or trades between the blocks. Neighboring bars are alike, so a shuffled test grades the strategy on days that look like the ones it was tuned on (scikit-learn’s notes on time-series data explain why).
- Write down what counts as a pass, now. Before any result, record the minimum number of trades, the average trade after costs you need, the deepest drawdown (fall from a high) you’d accept and how many versions you’ll allow yourself. A bar set after seeing the results tends to land just below them. Our guide on how many trades a backtest needs helps with the first number.
- Set costs and fills you could really get. Charge commission and exchange fees on every round trip (the entry and the exit), plus slippage: the ticks, or smallest price steps, you lose between the price you wanted and the price you got. Then set fills by order type:
| Order | Fills at | Why |
|---|---|---|
| Triggered by a bar’s close | The next bar’s open | Filling inside that bar is look-ahead bias |
| A stop order already resting when the bar opens | The stop price plus slippage, or the open if price gaps past it | Price can jump past a stop |
| A limit order | Only if price trades through it | A touch doesn’t mean your order was filled |
| Stop and target in the same bar | Assume the stop came first | Inside one bar, you can’t tell which price traded first |
- Run it on the build block, and log every version. Each change to a rule or a setting is a new version. Give each one a line in the log: its number, what changed, trades, average trade after costs, worst drawdown, and whether you kept it. Testing by hand? Keep everything after the current bar out of sight, and take every signal, including the ones that look like losers.
- Read the trades before the totals. Open a handful of winners and losers on a chart and check that each one did what your rules say. Then read the numbers.
- Run the finished version on the check block, once. Change nothing between the last build run and this one. If it fails here, the likeliest reading is that the build result was mostly fit to noise.
- Open the sealed block last, and only once. Run the exact version that passed the check. If you look, change something and look again, the sealed block has turned into more build data (why a sealed test only works once).
Even a sealed block is only one stretch of history. In a later paper, the same authors found that a single held-back block like this is an unreliable guard on its own (The Probability of Backtest Overfitting). That’s why the version log in step 5 matters as much as the split. Walk-forward analysis goes further: it repeats the build-and-check cycle across many windows of history.
Manual backtesting, a spreadsheet or code: which should you use?
Manual backtesting suits rules that need your eye, a spreadsheet suits simple rules on daily or hourly bars, and code suits anything with many trades or settings. Each has its own trap.
- By hand, with a chart’s replay mode. Good for rules that are hard to code and for learning a setup. The trap: it’s slow, samples stay small, and it’s tempting to skip trades you can tell will lose.
- A spreadsheet. Good for simple rules on daily or hourly bars. The trap: formula slips, and using a bar’s close before the bar has ended.
- Code, with free open-source libraries. Good for many trades and precise rules. The trap: bugs, and trying hundreds of versions without counting them.
- Backtesting software. Good for speed without coding. The trap: costs and fills you can’t see, so check both, and check that it shows you every trade.
Whichever you pick, the eight steps stay the same, and the tool only changes how fast you can make the mistakes.
For S&P 500 and Nasdaq-100 futures traders, Kwaant is another route: you describe your strategy in plain English, and it builds the test and shows you the trades, so you can check it understood your rules before anything is judged.
What goes in a backtest plan? A filled-in example
A test plan is one page you fill in before the first run: the rules, the data, the split, the costs, the pass bar and a limit on versions. Here is one for a made-up strategy, with invented numbers.
| Part of the plan | Made-up example |
|---|---|
| Rules | Buy at the next bar’s open after the first 5-minute close above the first 30 minutes’ high; one trade a day; no new entries after 12:00 exchange time; stop at that range’s low; target twice the risk; one contract; out at the close |
| Data | 6 years of 5-minute bars on one futures market, joined contract to contract and back-adjusted so the rolls don’t leave jumps |
| Split | Years 1 to 4 build, year 5 check, year 6 sealed |
| Costs | $4 in commission and fees, plus 1 tick of slippage each side at $10 a tick (2 ticks, $20): $24 a round trip |
| Fills | Entry at the next bar’s open after the signal bar closes; stop first if a bar hits both |
| Pass bar | At least 300 build trades, plus enough for the average to stand out from luck (worked out under reading the results); average trade above $0 after costs in every block; drawdown under $5,000 |
| Version budget | 20, every one logged |
You fix the luck rule now, but you can only work out its number after the build run, because it depends on the build’s average trade. The costs row is easy to leave blank, and it can decide the whole result.
When the costs per trade are bigger than the edge, the same trades go from a climb to a decline:
The version log for the same made-up strategy starts like this, one line per run:
| Version | What changed | Trades | Average after costs | Worst drawdown | Kept? |
|---|---|---|---|---|---|
| 1 | Rules as written | 500 | $6 | $3,800 | Yes |
| 2 | Stop at the middle of the range | 500 | $4 | $3,100 | No |
| 3 | Target three times the risk | 500 | $5 | $4,600 | No |
Three lines in, 17 versions of the budget are left, and version 1 has already had two rivals to beat.
How do you read backtest results?
To read backtest results, check the trade count first, then the average trade after costs, then the maximum drawdown; a win rate means little on its own. Together they’re evidence about one set of rules on one stretch of history.
| Number | What it tells you | Watch out for |
|---|---|---|
| Number of trades | How much evidence there is | Fewer than your plan said: luck can’t be ruled out |
| Average trade after costs (expectancy) | What the rules made per trade, net | A small average can vanish with slightly higher costs |
| Win rate | The share of trades that win | Means nothing without the average win and average loss |
| Profit factor | Gross profit divided by gross loss; above 1, the winners outweighed the losers | Easy to push up by tuning |
| Maximum drawdown | The deepest fall from a high | It’s one path; a future drawdown can be deeper |
| Results by year | Whether the profit is spread out | One great year carrying all the rest |
In the plan’s made-up strategy, 40% of build trades win, averaging $180, and 60% lose, averaging $70. That’s 0.4 × $180 − 0.6 × $70 = $30 a trade before costs, and $6 after the $24 round trip. The 40% win rate alone would have told you nothing.
Then the plan’s luck rule has its say. After costs, wins average $156 and losses $94, so a win lands $150 above the $6 average and a loss $100 below it. The spread per trade (how far a typical trade lands from the average) is √(0.4 × 150² + 0.6 × 100²) ≈ $122. A rough rule from statistics puts the trades needed at about (1.64 × spread ÷ average)², where 1.64 sets the common 95% level (how many trades a backtest needs explains it). Here that’s (1.64 × 122 ÷ 6)² ≈ 1,100 trades. Treat that as a floor: it pretends every win and every loss is exactly average, and it’s for a single version; with the three already in the log, the bar rises to roughly 1,900 (why every version raises the bar). The build block has 500, so the plan’s pass bar isn’t met. The verdict is “can’t tell yet”, and the check block stays unused, and the sealed block unopened, until a version has the trades to be judged.
Don’t lean on one headline number. In a 2016 study of 888 trading algorithms, Wiecki and colleagues found that the backtest’s Sharpe ratio explained less than 2.5% of the differences in how the algorithms did on new data (All that Glitters Is Not Gold). The same study found that the more backtests a developer ran on a strategy, the wider the gap between its backtest and its results on new data.
How much history do you need to backtest?
You need enough history to reach the trade count your pass bar sets, and to cover calm, volatile, trending and choppy markets. Count trades and market conditions, not years. In the made-up plan, the build block’s 500 trades came from four years, about 125 a year, so its 300-trade minimum alone needed about two and a half years. Even the single-version figure of about 1,100 trades would take about nine years at that pace.
- Trade count sets the minimum. A strategy that trades several times a day collects hundreds of trades a year. At two trades a month, 300 trades takes 12½ years.
- More versions need more history. Bailey and colleagues turn this into a minimum backtest length: the more versions you try, the longer the history has to be before the best one means anything.
- Each block needs variety. A strategy whose build years were one long trend has only been tested in a trend. Check that the check and sealed blocks hold some of the conditions the build block saw, and some it didn’t.
What should you do after a backtest?
After a backtest, the next step depends on which blocks it passed: a pass on the build block alone proves little, while a pass on the check and sealed blocks too earns a forward test, though never a promise.
| What you see | What it suggests | Usual next step |
|---|---|---|
| Passes on all three blocks | The edge held on data that didn’t shape it | Forward test it unchanged on new data before trusting it |
| Passes on build, fails on check | The build result was probably fit to noise | Record it as a no for this version; a new idea needs data its tuning hasn’t touched |
| Too few trades in any block | Can’t tell yet | Add history or markets, not more rules |
| Loses on build after costs | No edge in this history | Record it and move to the next idea |
Expect live results to be weaker than the backtest, especially one that was tuned. In a 2017 study of 215 strategies marketed to investors, Suhonen, Lennkh and Perez found a median drop of 73% in Sharpe ratio between backtest and live trading (Quantifying Backtest Overfitting in Alternative Beta Strategies). US futures rules require commodity trading advisors and pool operators to attach a warning when they show simulated results, and the rule’s own wording notes that simulated trading programs “are designed with the benefit of hindsight” (17 CFR 4.41). The usual gaps between a test and the market are covered in why backtests fail in live trading.
Whichever row you land in, keep the version log. It’s the one record that tells you, and anyone you show the result to, how many tries the winner had to beat.
Sources and further reading
- Bailey, Borwein, López de Prado and Zhu, Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance, Notices of the American Mathematical Society, 2014.
- Bailey, Borwein, López de Prado and Zhu, The Probability of Backtest Overfitting, Journal of Computational Finance, 2017.
- Wiecki, Campbell, Lent and Stauth, All that Glitters Is Not Gold, 2016.
- Suhonen, Lennkh and Perez, Quantifying Backtest Overfitting in Alternative Beta Strategies, Journal of Portfolio Management, 2017.
- Harvey and Liu, Backtesting, Journal of Portfolio Management, 2015: how to discount a backtested result for the number of tests behind it.
- 17 CFR 4.41: advertising by commodity pool operators and commodity trading advisors, Legal Information Institute.
- Cross-validation of time-series data, scikit-learn documentation.
- RealTest user guide, its page on joining futures contracts: why difference back-adjustment keeps point moves.
Questions people ask
How do you backtest a trading strategy for free?
Step through old charts with a charting platform’s replay mode and log each trade in a spreadsheet, or use a free open-source library if you code. The method itself costs only time. Good historical data, especially intraday futures data, is the part that often costs money.
Can you backtest a strategy without coding?
Yes. You can backtest by hand with a chart’s replay mode and a spreadsheet log, or use software that builds the test from settings or plain-English rules. Without code, the rules still have to be exact, and you still can’t let yourself see what happened after each entry before you take it.
How many trades do you need for a backtest?
There’s no single number, but there is a formula: roughly (1.64 × spread ÷ edge)² trades at the common 95% level for one version tested, where edge is the average result per trade and the spread per trade is how far a typical trade lands from that average. For example, a strategy averaging +0.25R a trade (R is the amount risked), with trades spreading about 1.25R, needs about 68 trades just to clear the bar, and roughly twice that to clear it reliably. Halve the edge, keeping the spread the same, and it needs about four times as many.
Can you backtest a discretionary strategy?
Only the parts you can write down as rules. Turn each judgment call into something a stranger could follow, such as a fixed time window or a measurable condition, and test that. If a call can’t be written down, a replay by hand tests you as much as the strategy.
What’s the difference between backtesting and forward testing?
A backtest runs your rules on past data. A forward test runs the finished rules on new data as it arrives, without changes, on paper or at small size. A forward test can’t be rushed: trades arrive at the market’s pace, so it takes far longer to collect the same number.
What are the most common backtesting mistakes?
Five mistakes cause many misleading backtests: costs left out or set too low, fills inside the signal bar, too few trades, versions tried but not counted, and judging a strategy on the same data used to tune it. Steps 2 to 5 above guard against each one.
General education about testing strategies, not investment advice. Figures are computer-drawn illustrations, not real market data.