Encoding (utf-8, latin-1, utf-16, etc.)
Delimiter (comma, tab, semicolon)
Quote character and escape behavior
Header row detection (present, absent, or duplicate)
Row count
Column count consistency across rows
Type inference per column (integer, float, date, string, mixed)
Null distribution per column
Duplicate row count

Read the output carefully before doing anything else. The diagnosis tells you which fixes apply.

Phase 2: Clean

Apply fixes in this order:

Fix encoding: re-read with detected encoding, write as utf-8.
Normalize line endings to \n.
Strip BOM and zero-width characters.
Fix headers: lowercase, snake_case, replace spaces with underscores, ensure uniqueness.
Trim whitespace from every cell.
Coerce types where the column is type-consistent.
Standardize null representations: empty string, "NA", "null", "n/a" all become empty.
Deduplicate exact rows.
Validate column count consistency. Flag rows that do not match the header.

Do not silently change data. If a fix alters values, report the count.

Output

Save the cleaned file as <original-name>-cleaned.csv in the same directory. Always utf-8, always comma-delimited, always with a header row.

Then emit a summary:

## Diagnosis
<key findings from phase 1>

## Fixes applied
- <fix>: <count of rows or cells affected>
- <fix>: <count>

## Manual review needed
- <issue that the script could not fix safely>

When to refuse

If the file appears to be a different format (Excel binary, JSON, fixed-width), do not try to clean it as CSV. Tell the user what the file actually is and suggest the right tool.

If the file has fewer than 5 rows after deduplication, ask the user whether the input is correct before continuing.

csv-cleaner

CSV Cleaner

Phase 1: Diagnose