test-run-rules
Test Run Rules
Apply these rules for the rest of the test run, whether it drives the running app or runs a test target. A scenario they leave blocked could not be set up or run; one they leave inconclusive ran, but its result cannot be read.
Sandbox Denials
When a command failed on a sandbox denial, whether it starts or reaches infrastructure or is a scenario's own command, re-run it via the Bash tool (dangerouslyDisableSandbox: true) before treating the infrastructure as unavailable or recording the scenario as failed, blocked, or inconclusive.
Isolation
- Isolate shared process state so concurrent or subagent runs don't collide: bind dev servers and services to unique ports, scope tmux sessions (
tmux -L <name>), give each browser session a unique name so cleanup can target only its own, and write screenshots and other scratch state to absolute paths under a uniquely named subdirectory of the session scratchpad directory. Derive each such identifier once and reuse that exact value in every later command, writing it as a literal or reading it back from a note in that subdirectory. A value recomputed per shell, such as$$, differs between the command that creates a resource and the command that releases it, so cleanup releases something it never created and reports success while the real resource leaks. A port picked as unique may already be held by a concurrent agent, so check it before binding and move to another when it is taken, leaving the incumbent running. When a unique port moves a service off its default address, find the settings elsewhere in the stack that name that default, such as allowed origins and sign-in callback URLs, and bring each in line through runtime overrides, leaving the working tree unchanged: add the new address beside the default in a list, and replace the default only where no process outside this run reads the setting. - Reuse a running dev server only when this session started it. Otherwise start one on a port this run selected and wait for it to be ready. Confirm it bound to that port before sending it traffic — a failed bind leaves another agent's service answering. Move to another port when the port is taken; report the error and stop when the server itself failed to start.
- Run each test runner this run starts in its own process group under a timeout enforced from outside the runner.
Unavailable Infrastructure
When a scenario needs infrastructure that cannot be stood up in this session (backend service, auth provider, external dependency), look for a stub before treating the scenario as blocked. When the dependency is reached through a client whose endpoint is runtime configuration, repoint that endpoint at a local stub; the same interception yields an artifact the system emits rather than provides (a token, a session identifier, a single-use link). Confine this to runtime configuration and leave the working tree unchanged. When the code under test makes the call itself, a call-site stub intercepts it without a second process: from the run, install a replacement into the running process that matches one destination and returns the response the scenario needs or delays the real one, gated on a flag the run can flip. Installing it from the run leaves the working tree unchanged as well. It lasts only as long as the process, so re-install it after anything that restarts or reloads it. When no stub applies or the ones that do fail, the scenario is blocked: name what is missing and what was tried.