eval-author-first-eval
Pass
Audited by Gen Agent Trust Hub on Oct 24, 2026
Risk Level: SAFEINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTIONCOMMAND_EXECUTION
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill processes untrusted external data, such as agent entry points, tool definitions, usage documentation, and ATIF traces, to generate evaluation cases and grading criteria.
- Ingestion points: Data enters the context during Stage 4 (Understanding the evaluation starting point) and Stage 5 (Prepare cases and grading) when reading repository files and agent documentation mentioned in
SKILL.md. - Boundary markers: The instructions do not specify explicit delimiters or 'ignore embedded instructions' markers when interpolating untrusted documentation into the case planning process.
- Capability inventory: The skill utilizes
Bash,Write,Read,Grep, andGlobtools across all stages to create and execute evaluation tasks. - Sanitization: No explicit sanitization or validation of the ingested natural language documentation is mentioned before it is used to influence the generation of evaluation logic.
- [DYNAMIC_EXECUTION]: The skill generates executable content at runtime, specifically NeMo Gym environment modules and Harbor task scripts (
test.sh,solve.sh). - Evidence: The skill uses
nemo_gym.environment.scaffold.scaffold_environmentto create Python modules and populates Harbor templates as described inreferences/gym-first-eval.mdandreferences/harbor-first-eval.md. - Severity Context: This is categorized as low severity because the code generation follows established templates for evaluation purposes.
- [COMMAND_EXECUTION]: The skill executes various system-level commands through the
Bashtool to manage evaluations and environments. - Evidence: Instructions include running
gym env validate,gym env test,harbor trial start,harbor job start, anddocker image rmto validate and run evaluations as documented in the reference files.
Audit Metadata