ml4t-validate-data
Installation
SKILL.md
Validate Data
Unvalidated data silently corrupts models - a single unadjusted stock split can make a momentum signal look 10x stronger than reality.
The Problem
Financial data arrives with missing values, duplicate timestamps, unadjusted corporate actions, stale prices, and impossible OHLC relationships. Using raw data without checks means your model trains on artifacts. A 50% overnight return that is actually a 2:1 split will dominate any feature that touches price changes. You will not see this in your loss function - the model happily fits the noise.
The Pattern
WRONG
import polars as pl
# Trust the data, start modeling immediately
df = pl.read_parquet("prices.parquet")
features = df.with_columns(ret=pl.col("close").pct_change())
model.fit(features) # Trained on splits, gaps, duplicates