For Business

Backtesting & Statistics

What Is a Sample Size

A sample size is the number of trades behind an answer, and a small one says what those trades did rather than what the rule does; doubt shrinks only with the square root of the count. This paper shows you sample size in five lines; the idea as a ladle from a pot; the running average of the plain 10:00 to 18:00 rule on the euro through 2026, trade by trade, thrashing early and settling late inside a funnel; the twenty-seven five-trade averages of the same rule, landing anywhere from minus 28 to plus 22; and the date range on your platform, which is what sets the sample.

Difficulty
Intermediate
Reading time
5 min
Pages
8
Reviewed by
Saman Rahbar

This paper is locked

Read it in the browser and keep the PDF with Premium, or buy this one paper.

Buy this paper — $2Get Premium

Sample size in five lines

A sample size is the number of trades behind a number, five or fifty or five hundred. It matters because a small sample says what those trades did, not what the rule does, and the help it gives grows only with the square root: four times the trades, half the doubt. The sign of too few is an average that keeps changing as trades are added. In the example, the plain 10:00 to 18:00 rule on the euro through 2026 averaged minus 27.9 pips a trade after five trades, where a pip is the fourth decimal place of the price, plus 5.2 after fifty, and plus 1.1 after 135. None of the early answers was wrong; each was only early.

A ladle from the pot

Picture a ladle from a pot. The pot is every trade the rule will ever make; the ladle is the trades you have seen so far. One spoonful can come up all salt or all water, and it says what was in the spoon, not what is in the pot. Stir and take fifty spoonfuls, and they start to agree with each other and with the pot, though never perfectly, because the pot is bigger than any number of spoonfuls. A five-trade result is one spoonful, and the honest thing to say about it is what it is.

This paper is locked

Read next