Data Resiliency Testing And Failure Injection

Overview

Use this skill when the goal is to prove that a data system recovers safely under failure, not only when everything goes right. It helps agents design controlled drills for retries, restarts, dependency outages, state recovery, duplicate prevention, backlog catch-up, and publish protection.

When to Use

• hardening a production pipeline before broad rollout
• testing failover, restart, replay, or checkpoint recovery behavior
• validating that retries do not duplicate or corrupt data
• proving recovery objectives for orchestrators, jobs, streams, or warehouse publishes
• converting a past incident into a repeatable resilience drill

Do not treat resilience testing as random breakage. The point is to validate recovery behavior with explicit safety limits and evidence.

Workflow