agentsop-idempotent-ingestion
Idempotent Ingestion · Re-Ingest-Correctness SOP
Third-person operating model for a coder agent that owns ingestion correctness across repeated runs. The audience is the LLM agent writing or reviewing the pipeline code — not the end user.
One sentence: The interesting run is the second one. A correct pipeline hashes each document, looks the hash up in a docstore, and inserts / updates / skips accordingly — so re-running over unchanged docs changes nothing.
This skill is an ENHANCE overlay over [[llamaindex]]. The base skill names
IngestionPipeline with docstore + UPSERTS_AND_DELETE as a hardening
must-have (OP-08, anti-pattern A10) but stops at "use it". This overlay
surfaces the correctness contract — what idempotency actually means, how the
hash decides the action, and how it generalises beyond LlamaIndex.