Feature Store Design

Feature stores exist to solve three problems at once: reuse across models, point-in-time correctness during training, and consistency between training and serving. A design that solves only one is incomplete - and the failure mode is expensive: silent label leakage that inflates offline metrics, or training-serving skew that makes a model score 0.85 offline and behave randomly in production.

Operating procedure

Step 1: Decide whether a feature store is warranted

Collect: number of models in production or planned, how many features they would share, and whether any model serves online. Rule of thumb: below 2-3 models sharing features, a feature store is premature infrastructure - a documented feature pipeline with a catalog (see ml-feature-engineering) does the job. Build or adopt one when features are shared across 3+ models, when online serving needs the same values training used, or when point-in-time bugs have already burned the team. Label the decision and its inputs; revisit when the model count doubles.

Step 2: Fix entity and naming contracts

Consistent naming is load-bearing - it is the API contract for every downstream model.

• Entity format: _id (e.g., user_id, item_id, session_id).
• Feature format: __ (e.g., user__payments__sum__30d). The name alone should tell a reader what the value is without opening the code.
• No abbreviations that are not defined in a project glossary.
• Each feature has exactly one owning team or owner string in metadata; unowned features are undeleteable and untrustable.