SHAP Analysis

Built-in feature_importances_ in tree models is biased toward high-cardinality and correlated features. SHAP values provide additive, consistent feature attributions grounded in game theory.

The Problem

LightGBM's feature_importances_ (gain or split-based) systematically overweights features with more unique values and features that are correlated with other predictors. Two features with identical predictive power but different cardinality will show different importance. This leads to incorrect feature selection, misleading model narratives, and poor decisions about which signals to keep or discard. SHAP values are the only feature attribution method that satisfies both local accuracy (attributions sum to prediction) and consistency (a feature that contributes more never gets lower attribution).

The Pattern

WRONG

import lightgbm as lgb

model = lgb.LGBMRegressor().fit(X_train, y_train)