Test — Capture a Failing Case as a Repeatable Check

Definition of done: one test-suite item exists in Opik that reproduces the failure — its data carries the failing input (and the expected output when one is known), and it has one or two binary assertions that would have failed on the bad trace and pass on a correct one. The item is in a named suite scoped to the project, is confirmed by reading it back, and is ready for /opik-compare to run. If the case can't be captured, stop at the first genuine blocker and return exactly one next step. Writing a pytest file, or describing a test in prose, is not success.

Operate: extract the case from real trace data, write assertions a judge can decide with a yes or no, store them where /opik-compare will find them — and change no application code. This skill writes to Opik, never to the repo.

Inputs

The entry point is /opik-test <trace-id> (capture that trace), /opik-test <describe the failure> (find the trace, or capture from the description alone), or /opik-test right after /opik-explain (capture the trace it just explained). Infer the rest; treat these as optional overrides:

• suite name (default: <project>-regressions) · project (default: the trace's project, else the configured one) · expected output (default: none — assertions carry the expectation) · execution policy (default: the suite's; runs_per_item: 3, pass_threshold: 2 only for an intermittent failure).

Ask only at a genuine, non-inferable blocker (see Blockers).

Activation — the only in-scope work