Feature Selection

Selecting features on the full dataset is a form of lookahead bias. The test set influences which features are kept, inflating out-of-sample performance. Feature selection must happen inside each CV fold.

The Problem

With 50 features and a finite sample, some will correlate with forward returns by chance alone. If you rank features by IC on the full dataset and keep the top 10, those 10 are partly selected for noise. Out-of-sample, the noise component vanishes and the model underperforms expectations.

The Pattern

WRONG

from scipy.stats import spearmanr
import numpy as np

# Feature selection on full dataset - leaks test-set information
ic_scores = {col: abs(spearmanr(X[col], y).statistic) for col in X.columns}
selected = sorted(ic_scores, key=ic_scores.get, reverse=True)[:20]
model.fit(X[selected], y)