For Business

Backtesting & Statistics

What Is Out-of-Sample Testing

Out-of-sample testing sets some bars aside before a rule's numbers are chosen, fits the rule on the rest, and then runs it once over the set-aside bars, which are the only ones it could not have been bent to fit. This paper shows you out-of-sample in five lines; the idea as a fence across a field; three weeks of the euro in August 2026 on real bars, with the hours fitted on the first two weeks and tested on the third; the two numbers as two circles; and the Forward setting in the Strategy Tester on your platform, which fences the last part of a test off for you.

Difficulty
Beginner
Reading time
5 min
Pages
8
Reviewed by
Saman Rahbar

This paper is locked

Read it in the browser and keep the PDF with Premium, or buy this one paper.

Buy this paper — $1Get Premium

Out-of-sample in five lines

The difference this paper is about is between the bars a rule was built on and the bars it was not. In-sample bars are the ones you looked at while choosing the rule's numbers; out-of-sample bars are set aside before you start, and the rule meets them once, at the end. The reason is that a rule fitted to bars it has seen will look good on them, and that proves nothing. The method is to split the tape, fit on the left, test once on the right, and not go back. In the example, the best buy-and-sell hours on the euro for 3 to 14 August 2026, buy at 07:00 and sell at 16:00, made 123.7 pips there, where a pip is the fourth decimal place of the price; on the week set aside, 17 to 21 August, they made 75.1.

A fence across the field

Picture a fence across a field. The left half is ploughed: every row has been tried, the rule's numbers have been chosen and changed and chosen again on those bars. The right half is untouched, fenced off before the ploughing began, and the rule crosses the fence once, when it is finished. The order matters entirely. A fence put up afterwards, around the part where the rule happened to do well, is not a fence; it is a line drawn round the good part, and it tells you nothing.

This paper is locked

Read next