run-evals

Installation
SKILL.md

This skill orchestrates the agent-eval suite: natural-language cases executed by a headless claude -p agent using the bundled skill (skills/sim-use/) against the Playground fixture apps, judged by deterministic post-condition checks. It verifies the layer the scripted E2E suites cannot: that an agent reading SKILL.md reaches for the right verbs and survives the documented pitfalls. A failure here with a green scripted layer usually means skill-prose drift, not a CLI bug.

Execution is delegated to scripts/eval.sh / e2e/agent-evals/run.py — do not reimplement their logic. Case anatomy, tags, and authoring rules live in e2e/agent-evals/README.md. Run from the repo root.

Step 1: Decide WHICH sim-use is under test

The whole run — device probing, the agent's commands, the verification layer — resolves sim-use from PATH unless overridden. Never let this be implicit:

Installs
1
GitHub Stars
1.4K
First Seen
Aug 15, 2026
run-evals — lycorp-jp/sim-use