feature-engineering

Overview

Based on "Feature Engineering for Machine Learning" by Alice Zheng and Amanda Casari. The core principle: features are the interface between raw data and model performance. The right transformation of an existing variable outperforms adding more data or tuning hyperparameters in most real-world settings. Feature engineering is domain knowledge encoded into math - and it is the highest-leverage step in the ML pipeline.

Workflow

Step 1: Audit raw columns for engineering opportunity

Before transforming anything, catalog what you have and what problems each column has.

For each column, note:

• Type: numeric continuous, numeric discrete, ordinal categorical, nominal categorical, datetime, text, ID
• Problem: skewed, high cardinality, missing, mixed type, free text, leaky (contains target signal from the future)
• Engineering opportunity: log transform, binning, encoding, extraction, embedding

Create a feature engineering plan as a table before writing any code: