The Agent Skills Directory

[DYNAMIC_EXECUTION]: The skill dynamically generates Python scripts for evaluation graders and unit tests based on data parsed from the goldset.json file. These scripts are written to evals/{system}/graders/ and evals/{system}/tests/ at runtime.\n- [COMMAND_EXECUTION]: The skill automatically executes the generated Python code using the pytest command as part of its 'Grader Calibration Loop' (Phase 2b). This execution occurs immediately after code generation to verify correctness.\n- [INDIRECT_PROMPT_INJECTION]: There is a risk of indirect prompt injection as the code generation process consumes untrusted data from goldset.json. Malicious content in the pass_condition, fail_condition, or examples fields could influence the generated Python logic to perform unintended or harmful actions.\n
Ingestion points: The skill reads evals/{system}/goldset.json (SKILL.md).\n
Boundary markers: No boundary markers or specialized delimiters are used to isolate untrusted data from the code generation logic.\n
Capability inventory: The skill can perform file writes, execute shell commands (pytest), and run Git operations via setup scripts.\n
Sanitization: The process flow does not describe any sanitization or validation of the JSON inputs before they are interpolated into executable Python files.

evals-implement