Latent Factor Extraction

PCA explains 80% of return variance - but variance is not alpha. The first principal component captures market beta, which earns the equity premium, not a tradeable edge. Confusing variance-explained with pricing power is the central mistake.

The Problem

With 400+ published return predictors, hand-picking factors invites overfitting. Latent factor methods (PCA, autoencoders) extract structure directly from data. But three failure modes undermine them:

1. Variance != pricing - high-variance factors may capture idiosyncratic noise, not compensated risk.
2. Eigenvector instability - when assets (N) approach time periods (T), sample covariance is dominated by noise. Marchenko-Pastur theory gives the noise boundary.
3. Full-sample PCA is leakage - fitting PCA on the complete panel, then testing on a held-out period, leaks the covariance structure of the test period into training.

The Pattern

WRONG

from sklearn.decomposition import PCA
import numpy as np