staff-engineering-skills-object-store-as-database
Installation
SKILL.md
Object Store as Database
S3 is no longer just a blob store. With strong read-after-write consistency (2020) and conditional writes (2024), S3 is a linearizable key-value store for single-key operations. The trap is not "don't use S3 as a database" -- it's using S3 as a database without using conditional writes for correctness.
What Changed
| Year | Change | Impact |
|---|---|---|
| 2020 | Strong read-after-write consistency | Read after write returns latest; LIST reflects current state. |
| Aug 2024 | If-None-Match: * (put-if-absent) |
Atomic create-if-not-exists. Enables append-only logs, idempotent writes. |
| Nov 2024 | If-Match: <etag> (compare-and-swap) |
True CAS on objects. Enables optimistic concurrency, leader election, metadata catalogs. |
Together these give linearizable single-key operations: read an object (get its ETag), compute a new value, write it back with If-Match. This is the building block under Delta Lake, Iceberg, SlateDB, WarpStream, Chroma, and dozens of others.
The Fundamental Pattern
Almost every system using S3 as a database follows the same architecture: