Dedupe — Data Deduplication Reference

Quick-reference skill for deduplication strategies, algorithms, and data quality patterns.

When to Use

• Removing duplicate rows from datasets or databases
• Deduplicating files in storage systems
• Implementing fuzzy matching for near-duplicate detection
• Choosing between exact and probabilistic dedup methods
• Building ETL pipelines with deduplication stages

Commands

intro

scripts/script.sh intro