staff-engineering-skills-sharding
Installation
SKILL.md
Sharding Trap
Sharding is nearly impossible to undo and fundamentally changes what your code can do. Before sharding, ask: have you exhausted every simpler alternative?
Do You Actually Need to Shard?
Almost certainly not. Work through this checklist first:
| Step | Solution | Handles |
|---|---|---|
| 1 | Indexes -- run EXPLAIN ANALYZE on slow queries |
Missing indexes cause 90% of "database is slow" |
| 2 | Query optimization -- fix N+1 queries, unnecessary JOINs, full scans | Bad queries, not database limits |
| 3 | Connection pooling -- PgBouncer or built-in pooling | Connection exhaustion |
| 4 | Caching -- Redis for hot data, query result caches | Read throughput |
| 5 | Read replicas -- route reads to replicas | Read scaling |
| 6 | Table partitioning -- partition by date range within one database | Large table performance, transparent to app code |
| 7 | Vertical scaling -- bigger machine (64-core, 256GB RAM) | Everything, up to a point |
Only if ALL of these are insufficient should you consider sharding. A single PostgreSQL instance with proper indexing and read replicas handles far more than most people expect. If you have less than 1TB of data or fewer than 10,000 writes per second, you almost certainly don't need to shard.