Data Migration Best Practices

Treat migration as a temporary production system with explicit correctness, capacity, observability, and retirement criteria. It changes persisted behavior and is not automatically a refactoring.

Workflow

1. Define source authority, target contract, transformations, invariants, data classification, and tolerated downtime/staleness.
2. Choose a stable snapshot boundary or high-water mark and start durable capture of live changes before they can escape the snapshot.
3. Expand the target schema compatibly; keep old and new application versions interoperable during rolling deployment.
4. Backfill in bounded, idempotent, resumable batches with deterministic progress and explicit bad-row handling.
5. Apply captured deltas without allowing stale snapshot data to overwrite newer writes.
6. Reconcile availability, counts, identities, values, and domain invariants against a declared watermark.
7. Cut over only when freshness and discrepancy gates pass; monitor and retain a reversal or forward-repair path.
8. Contract old fields, code paths, capture infrastructure, and temporary permissions only after an observation window.

Decision Rules