How Many Backtests Do You Need Before Trusting a Strategy?

Most traders backtest twenty setups, see a 65% win rate, and decide they've found something. They haven't. They've found noise wearing a costume. Twenty trades can't tell the difference between a real edge and a lucky streak — and the confidence that comes from that tiny sample is exactly what blows accounts.

So how many backtests do you actually need before you trust a strategy? The honest answer has a number attached, and it's bigger than you'd like. Here's the real math on backtesting sample size, why small samples lie, and how much data it takes before your win rate means anything.

How many backtests is "enough"?

Short version: aim for at least 100 logged setups to see a signal, and 300–400 before you'd bet real size on it — spread across different market conditions, not clustered in one clean trend. Below 100, your numbers are mostly random. Below 30, they're meaningless.

That's the rule of thumb. The reason behind it is where it gets interesting, because once you see why, you stop trusting small samples for good.

Why 20 trades lies to you

Here's the uncomfortable part. Flip a fair coin — a genuine 50/50, zero edge — twenty times, and it'll come up heads 13 or more times about one in eight attempts. Thirteen out of twenty is a 65% "win rate." From a coin. From nothing.

So if you test twenty setups and see 65%, you've learned almost nothing, because a worthless strategy produces that exact result roughly one time in eight. Run three or four strategies with no edge at all and one of them will look like a winner. That's not a strategy — that's survivorship bias with a spreadsheet.

The statistics behind this are boring and unforgiving. The uncertainty in a win rate scales with the square root of your sample size. At 20 trades, a measured 50% win rate carries a 95% confidence interval of roughly 28% to 72%. That's not a rounding error — that's the difference between a great system and a blown account, and 20 trades can't tell you which one you have.

What sample size actually buys you

More trades don't just make you feel better — they physically shrink the range of what your true win rate could be. Take a setup that's genuinely a 50% winner and watch the 95% confidence interval tighten as the sample grows:

• 20 trades: true win rate could be anywhere from ~28% to ~72%
• 100 trades: ~40% to ~60%
• 400 trades: ~45% to ~55%

Notice what happened between 100 and 400. To halve the width of that error band, you needed roughly four times the trades — because the math runs on the square root of the sample, not the sample itself. That's the single most important thing to understand about backtesting sample size: getting from "rough idea" to "actually reliable" isn't a little more work, it's a lot more. It's also exactly why serious testing needs years of data, not a handful of recent sessions.

This matters most when your edge is thin. If your strategy lives or dies on the difference between a 52% and a 48% win rate — as plenty of real, profitable systems do — then a 100-trade sample whose true value could sit anywhere from 40% to 60% literally cannot tell you whether you have an edge or a slow bleed. You need the tighter band, and the tighter band needs the bigger sample.

Which raises the obvious question this article has so far dodged: what does a bigger sample actually cost you in hours? It is more knowable than most people assume, and usually smaller — the time arithmetic of a manual backtest works it out, and the dominant variable turns out not to be the number of trades at all.

It's not just how many — it's how varied

Sample size isn't only a count. Four hundred trades all taken inside one smooth uptrend is a worse test than 150 trades spread across a rally, a correction, a choppy range, and a news-driven week. Markets change character. A setup that prints money in a trend can quietly hand it all back in a range, and if your entire backtest happened during the trend, you've measured one regime and assumed it's the whole market.

This is the real problem with testing a couple of weeks of data. It's not just that two weeks gives you too few setups — though it usually does. It's that two weeks is almost always one market condition. You cannot see how a strategy handles a correction if there wasn't one in your window. How much data do you need to backtest a strategy properly? Enough to contain the conditions you'll actually trade through — which, in practice, means years, not weeks. A year of history at minimum; several years if you want to see how your edge behaves across genuinely different environments.

The two failure modes stack. Too few trades and the numbers are statistically meaningless. Too little variety and they're meaningful but only describe a world that no longer exists.

There's a third failure mode that size can't fix, and it's worth naming here because a large sample makes it more convincing rather than less: if you refined the rules while looking at the same data you're measuring, the result describes your hindsight, not your edge. That's what out-of-sample testing is for — hold back a block of history, freeze the rules, and test on data that had no part in producing them. Sample size decides whether your number is stable; out-of-sample decides whether it's measuring anything at all.

How many trades to trust an edge (not just a win rate)

Win rate is the easy half. What actually determines whether you're profitable is expectancy — your average win rate combined with your average risk-to-reward — and proving expectancy is reliably positive takes more trades than proving a win rate, because you're now measuring two noisy things at once.

The rougher your edge relative to your trade-to-trade swings, the more trades you need to see it through the noise. A strategy with a big, obvious edge might reveal itself in 100–150 trades. A thin edge — the more common case — can need several hundred before the profit clearly separates from randomness. There's no single magic number, because it depends on your strategy's variance, but the direction is always the same: thinner edge, bigger sample. When in doubt, assume your edge is thinner than it looks and test more.

You don't have to estimate that by feel. The backtest sample size calculator takes your win rate, your reward-to-risk and the number of trades you've logged, and returns the specific figure for your strategy: the range your true expectancy could occupy, how often a strategy with no edge at all would score as well as yours did, and how many more trades it would take before the answer stops being ambiguous. It's the same math as this section, run on your numbers instead of an example's.

If you want to see what a thin edge costs you in practice, the prop firm challenge calculator makes it concrete. Drop a 45% win rate at 1:2 to 33% — still a respectable-sounding number — and your expectancy goes to zero, turning a challenge that passes comfortably into a coin flip. That gap is exactly the thing a sample too small to measure will hide from you.

How to actually get a sample worth trusting

Sample size is one stage of a larger procedure — if the rest of your method is still loose, how to backtest a trading strategy covers writing testable rules, replaying bar by bar and validating out of sample, with this page as its sample-size stage.

Knowing you need a few hundred varied setups is one thing. Getting them without fooling yourself is another. The process is the same one that underpins any honest test:

1. Write the rules down before you test. If the entry is fuzzy, every result is fuzzy. A precise, if-this-then-that rule is the only thing you can count honestly.
2. Replay candle by candle. Step through history one bar at a time so you only ever see what you'd have seen live. The moment you scroll ahead, hindsight inflates a 45% strategy into a 90% fantasy — and this is the fastest way to fake a big sample that means nothing.
3. Test across real history, not the last fortnight. Pull months to years of data so your sample actually contains trends, ranges, and news — the conditions you'll trade through.
4. Log every valid setup, including the ugly ones. Cherry-picking the clean winners is how a losing system looks like a winner. The log is the edge.
5. Judge the sample, not the trade. One trade is noise. A hundred is a signal. A few hundred across varied conditions is conviction.

If you trade Candle Range Theory or ICT, the mechanics of running these tests are laid out step by step in How to Backtest the CRT Strategy and How to Backtest ICT Concepts — and if you're still on the concept itself, start with What Is CRT (Candle Range Theory)?.

The catch is data. You can't build a 300-setup sample across multiple market regimes on two weeks of price. That's the whole reason CRTLAB gives you real historical data by the year — replay a market candle by candle, mark your setups, and stack up a sample big enough to actually trust. Start free, and stop mistaking a lucky twenty for an edge.

FAQ

How many backtests do I need before trusting a strategy? As a working rule, at least 100 logged setups to see a real signal and 300–400 before you'd risk meaningful size — and when you do scale up, let the position size calculator set the number rather than your confidence — spread across different market conditions. Fewer than 30 and your results are statistically meaningless; a thin edge needs the higher end of that range.

How many trades is a good backtesting sample size? A few hundred is the practical target. At 20 trades a measured win rate could be off by more than 20 percentage points in either direction; at 400 the error band narrows to roughly ±5%. Because uncertainty falls with the square root of the sample, halving your error requires roughly four times the trades.

Can you trust a backtest with 20 or 30 trades? No. A genuinely edgeless strategy produces a 65%+ win rate over 20 trades about one time in eight by pure chance. Small samples can't separate a real edge from a lucky streak, which is exactly why they lead to overconfidence.

How much data do I need to backtest a strategy? Enough history to contain the market conditions you'll actually trade — realistically years, not weeks. Two weeks of data usually means both too few setups and a single market regime, so it can't tell you how your strategy behaves in a correction, a range, or a news event.

Does a bigger sample guarantee my strategy works? No — it makes your measurement trustworthy, not your strategy profitable. A large, varied, honestly-logged sample gives you a win rate and expectancy you can believe. If those numbers are good, you've earned conviction. If they're bad, you found out for free instead of with your account.

The same sample-size logic applies in the other direction once you go live: twenty live trades is not enough evidence to abandon a strategy either, and a live number will sit somewhat below the tested one for reasons that have nothing to do with the edge failing.