How to backtest a seasonal pattern without fooling yourself

A seasonal backtest is unusually easy to get wrong, because the thing you are testing was found by looking at the same data you are testing it on. Everything below is about closing that loop.

The four failures, in order of cost

1. Look-ahead in the selection, not the trade

Almost everyone gets the trade right — enter on this date, exit on that one, using only prices available at the time. Then they select which pattern to trade using the full history, including the years they are about to test on.

If you pick the ten stocks with the best August record over 2005-2025 and then measure how those ten did in August 2005-2025, you have measured your own selection. The result will be excellent and it will mean nothing.

The fix is point-in-time selection: to decide what to trade in year Y, use only the years before Y. A rule qualified on 2005-2014 gets traded in 2015, then requalified for 2016, and so on. The number this produces is dramatically worse than the naive one. It is also the only one that corresponds to something you could have done.

2. Survivorship

Index membership lists are almost always the current list. Applied backwards, they quietly delete every company that went bankrupt, got acquired, or was dropped for poor performance — which is the same as testing a strategy on a universe pre-filtered for having survived.

You often cannot fix this without paid point-in-time constituent data. What you can do is say so, and treat the result as an upper bound rather than an estimate.

3. Multiple comparisons

Twelve months, six holding periods, five hundred stocks. That is thirty-six thousand combinations, and a few hundred of them will look spectacular on chance alone. Searching that space and reporting the winner is not research, it is a lottery with the losing tickets thrown away.

Two defences. First, decide the rule before searching, so there is only one test. Second, if you do search, hold out a block of years the search never saw and check the winner there. A pattern that fails out of sample was never a pattern.

4. Costs, and the size of the edge

Commissions, spread, slippage, and — if it is a taxable account — the difference between short and long term treatment. A seasonal window of a few weeks generates a lot of round trips per year, and an edge of half a percent per trade does not survive them.

Compute the cost per round trip in currency, not basis points, and put it beside the average gain per trade in the same units. If the second is not several times the first, stop.

The three numbers to report

An average return, on its own, is close to useless. Report these together or you have not described the pattern:

• Win rate. How many of the observations were positive. This separates a real tendency from one good year.
• Median. When the median sits far below the mean, the average is an artefact of outliers you will not reliably catch.
• Worst observation. The number that decides whether the position is holdable. A strategy you abandon in its worst year has the worst year's return and none of the good ones.

A fourth, if you are sizing a portfolio rather than a single trade: maximum drawdown on the equity curve, which is what actually ends strategies.

How many observations is enough

Fewer than you would like, and more than you have. A twenty-year lookback on an annual seasonal window gives you twenty observations. That is a small sample by any standard, and no amount of statistical treatment will make it larger.

The practical consequence: be suspicious of any seasonal claim with a win rate of exactly 100%. On twenty observations, "never failed" is a common outcome for patterns with no real edge at all.

Run the checklist on a real pattern

1. 1Pick a ticker and set the range to the full history available, not just the years you like.
2. 2In Trading Analysis, set the entry and exit dates of the window you want to test.
3. 3Record all four numbers: win rate, average, median, worst year. Not just the first one you like.
4. 4Halve the sample: run the same window on the older years only, then the newer years only. A real pattern shows up in both.
5. 5Subtract your round-trip cost in currency from the average gain per trade before deciding anything.

Step 4 is the one everybody skips, and it kills more patterns than the other four combined.

Open TradeSeasons

A checklist

Before trusting a seasonal rule, confirm every one of these:

1. Selection used only data available before each traded year.
2. The universe includes names that later disappeared, or the survivorship bias is stated.
3. The rule was fixed before the search, or validated on years the search never saw.
4. Round-trip costs are subtracted, in currency.
5. Win rate, median and worst case are reported alongside the average.
6. The sample size is stated, and it is not five.

Anything that fails one of those is a hypothesis, not a strategy.