Avoiding Backtesting Overfitting by Covariance-Penalties: An Empirical Investigation of the Ordinary and Total Least Squares Cases

Abstract

Systematic trading strategies are rule-based procedures that choose portfolios and allocate assets. To attain certain desired return profiles, quantitative strategists must determine a large array of trading parameters. Backtesting, the attempt to identify the appropriate parameters using historical data available, has been highly criticized because of its abundance of misleading results. Hence, there is increasing interest in devising procedures for the assessment and comparison of strategies (i.e., devising schemes for preventing what is known as backtesting overfitting). So far, many financial researchers have proposed different ways to tackle this problem that can be broadly categorized into three types: data snooping, overestimated performance, and cross-validation evaluation. In this article, the authors propose a new approach to dealing with financial overfitting, a covariance-penalty correction, in which a risk metric is lowered given the number of parameters and amount of data used to underpin a trading strategy. They outline the foundation and main results behind the covariance-penalty correction for trading strategies. After that, the authors pursue an empirical investigation and compare its performance with some other approaches in the realm of covariance-penalties across more than 1,300 assets, using ordinary and total least squares. Their results suggest that covariance-penalties are a suitable procedure to avoid backtesting overfitting, and total least squares provides superior performance when compared to ordinary least squares.

TOPICS: Statistical methods, simulations, big data/machine learning

Key Findings

• • A literature review of backtest overfitting, putting in perspective the different approaches available.

• • A new Covariance-Penalty formula to correct Sharpe ratios based on the number of parameters in a model.

• • An empirical investigation across 1300 assets, using ordinary and total least squares, comparing our technique with others from the Covariance-Penalty literature.