evals-implement
Warn
Audited by Gen Agent Trust Hub on Aug 4, 2026
Risk Level: MEDIUMCOMMAND_EXECUTIONREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
- [REMOTE_CODE_EXECUTION]: The skill dynamically generates Python grader scripts and unit tests in the
evals/directory based on contents fromgoldset.json. It implements a self-tuning mechanism in Phase 2b that edits and re-executes these scripts up to three times.\n- [COMMAND_EXECUTION]: The skill executes thepytestcommand to run the generated unit tests against the generated grader logic.\n- [PROMPT_INJECTION]: The skill exhibits an indirect prompt injection surface by ingesting external data to synthesize LLM-judge templates and code logic.\n - Ingestion points: Reads criteria and examples from
evals/{system}/goldset.jsonin Phase 1.\n - Boundary markers: No delimiters or instructions are used to prevent the agent from obeying instructions embedded within the goldset data.\n
- Capability inventory: Subprocess execution of
pytestand file system write access inevals/{system}/graders/andevals/{system}/tests/.\n - Sanitization: No validation or sanitization is applied to the goldset data before it is interpolated into executable code or prompts.
Audit Metadata