nemo-evaluator
Pass
Audited by Gen Agent Trust Hub on Sep 23, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
- [EXTERNAL_DOWNLOADS]: The skill instructs users to install the
nemo-evaluator-launcherpackage via pip and pull containers from NVIDIA's container registry (nvcr.io). These are well-known services used for their intended purpose in LLM evaluation workflows. - [COMMAND_EXECUTION]: The skill uses the
nemo-evaluator-launcherCLI tool to run benchmarks, check job status, and export results. These commands are consistent with the skill's primary purpose of model benchmarking. - [INDIRECT_PROMPT_INJECTION]: The skill is a benchmarking tool that ingests and evaluates outputs from external LLM APIs (e.g., NVIDIA Build, vLLM). This creates a surface where malicious instructions in model responses could attempt to influence the evaluation environment.
- Ingestion points: LLM API response bodies (completions/chat completions) in
SKILL.md. - Boundary markers: None explicitly defined in the skill instructions.
- Capability inventory: File system writes (
results/), network access to API endpoints, and shell command execution via the launcher. - Sanitization: The skill instructions do not specify sanitization, though the tool is documented to use containerized (Docker) execution to isolate benchmark runs.
Audit Metadata