AI Done: Definition of Done for AI Features

Traditional software is a vending machine: the same input gives the same output, and "all tests pass" means done. AI features break that. The same prompt produces different outputs across users, sessions, model updates, and contexts your QA never imagined. What you shipped is not a fixed object, it is a distribution of behaviors. So "done" cannot be a checkbox.

An AI feature is done when it is calibrated: you have decided the variance you will accept, planned for the failure modes, rehearsed the rollback, and wired the loop that catches the surprises that show up after launch.

"If 'done' was the language of a build culture, 'calibrated' is the language of a learning culture." (Jeff Gothelf, "What 'done' means when you're shipping AI features", https://jeffgothelf.com/blog/what-done-means-when-youre-shipping-ai-features/ , retrieved 2026-06-02)

This skill is a thin orchestrator. It does NOT re-implement what you already have:

• Deterministic pass/fail checklist: it calls /success-criteria, then the checklist loop.
• Probabilistic eval suite + grading: it calls /eval-harness.

It adds the three layers neither of those covers (failure-triage playbook, tripwires + a rehearsed rollback, post-ship eval loop), then stamps the global QA bar. If you find yourself hand-writing pass/fail checks or eval graders inside this skill, stop and call the owning skill. Staying thin is the point.

When to use (and when NOT to)

USE when the feature has any probabilistic / AI surface: an LLM call, model inference, an agent loop, a recommender, generated text / image / audio, or a classifier whose output a user sees.