run-agentic-tests
Installation
SKILL.md
Run Agentic Tests
Coordinate a real E2E campaign from scenarios to independently inspected evidence, scoped repairs, integrated-target retests, and an inspectable HTML report. This skill provides the operational harness for multi-agent testing across web, mobile, MCP, and CLI, integrating directly with specialized test runners.
Model selection & reasoning configuration per provider
Models are configured strictly per provider to balance execution speed, assertion accuracy, and token economics. Core architectural rule: Anthropic splits models by family (no reasoning level needed), while Codex and Gemini split capabilities primarily by reasoning effort.
| Provider | Task category | Model | Reasoning effort | Operational rule |
|---|---|---|---|---|
| Anthropic | All operations (execution, verification, diagnosis) | sonnet-5.5 or haiku-5.5 (use newer version if released) |
None | Anthropic sorts models by family; reasoning level is omitted (if runtime extended thinking is enabled, cap thinking budget to low/medium). Never use heavy flagship family models like Opus for automated loops. |
| Codex | Simple ops (routine execution, CLI commands, basic steps) | gpt-6-luna |
xhigh |
luna is a compact model; xhigh reasoning effort keeps tool calls and deterministic steps rock-solid. |
| Codex | Evidence checks (verifiers, assertion checks, artifact analysis) | gpt-6.1-sol |
medium |
Balanced reasoning ensures rigorous evaluation of screenshots, logs, and state evidence without timeout delays. |
| Gemini | Simple ops (routine execution, navigation, basic steps) | gemini-3.8-flash |
medium |
High throughput, fast response times for multi-step browser/CLI steps. |
| Gemini | Everything else (evidence checks, verification, diagnosis, planning) | gemini-3.8-flash |
high |
Increased reasoning depth for independent verification, root-cause diagnosis, and scenario authoring. |