The Agent Skills Directory

[COMMAND_EXECUTION]: The harness is designed to execute shell scripts provided within the task family structure, specifically hooks/preflight.sh and hooks/invariants.sh. These scripts run on the host machine to prepare the environment and validate results.
[EXTERNAL_DOWNLOADS]: The CLI supports using git URLs for the --family flag, which allows the tool to download benchmark task families from remote repositories. This involves subsequent execution of shell scripts contained within those downloaded repositories.
[DATA_EXPOSURE]: The tool automatically discovers and loads .env and .env.local files from the task directories into the environment. It includes a documented safety feature that adds discovered variable names to a redaction allowlist to prevent accidental leakage in agent traces.
[INDIRECT_PROMPT_INJECTION]: The skill processes external data in the form of task prompts (agent.task.md, judge.task.md) and renders them into templates. It defines a capability surface for the agent under test that includes Bash, Write, and Edit tools. However, the tool is specifically designed to isolate grading logic (hidden tests) from the agent's working directory to maintain evaluation integrity.

fit-benchmark