ml4t-non-stationarity
Installation
SKILL.md
Non-Stationarity
Financial time series have means, variances, and correlations that change over time. A model trained on 2015-2019 low-volatility data will underperform in a 2020 regime shift if it assumes fixed parameters.
The Problem
Global normalization (subtracting the full-sample mean and dividing by the full-sample standard deviation) embeds future information into every observation. It also assumes the distribution is stable, which is false for financial data. Post-2008 interest rates, COVID volatility, and factor decay are all examples of structural shifts that invalidate fixed-parameter assumptions.
A model trained on globally normalized features will overfit to the training regime and degrade when the regime changes.
The Pattern
WRONG
# Global normalization: uses future data and assumes stationarity
X_norm = (X - X.mean(axis=0)) / X.std(axis=0)