4 Examples of Backtesting

John Spacey, updated on April 15, 2017

Backtesting is the process of testing a strategy, model or hypothesis against historical time-series data. The following are common examples.

Investing

Backtesting is extremely common amongst individual investors with tools that can be used without an understanding of the underlying data science. For example, an investor might test a strategy such as buying a particular equity each time it hits a 52 week low over many decades. Such a test suffers from a survivorship bias but such flaws often go unnoticed.

Business Strategy

Backtesting business strategies against historical transactional data. For example, a retailer might look at the historical sales of women's shoes at a $60 price versus a $55 price. Business data often doesn't age well. For example, backtesting price strategy against 5 year old data is of questionable value.

Research

Backtesting is a common research method that has value when a disciplined approach is employed. For example, if a researcher has reason to believe that people who live close to major highways have a lower life expectancy, she might backtest the idea against multiple sets of data.

Data Dredging

Data dredging is looking for patterns in historical data using automated tools. This often detects random patterns that are meaningless. For example, a tool might discover that people who brush their teeth with a particular brand of toothpaste are more likely to purchase a red car. Data dredging can be used to produce questionable research and statistics with little effort.

Overview: Backtesting
Type	Data Science
Definition	The process of testing a strategy, model or hypothesis against historical time-series data.
Related Concepts	Data Science Data Dredging Data Mining Data Wrangling Misuse of Statistics Regression Analysis Survivorship Bias