nemo-evaluator-plugin
Pass
Audited by Gen Agent Trust Hub on Aug 20, 2026
Risk Level: SAFE
Full Analysis
- [COMMAND_EXECUTION]: The skill provides instructions for executing the
nemo evaluatorCLI tool. These commands are standard operations for managing evaluation jobs, such asinfo,metric-types,evaluate run, andevaluate submit. All provided commands are scoped to the intended purpose of the skill and do not include dangerous shell operations or unauthorized system modifications.- [CREDENTIALS_SAFE]: The skill demonstrates safe credential management. Inreferences/api-auth.mdandassets/specs/llm_as_judge.json, it instructs users to use environment variable names (e.g.,NVIDIA_API_KEY) or NeMo platform secret names instead of hardcoding raw API keys. The documentation explicitly warns against printing secrets to stdout.- [DATA_INGESTION]: The skill processes evaluation datasets in JSON or JSONL format as part of its primary function. While this presents an indirect prompt injection surface (Category 8), the risk is mitigated by the fact that the skill is intended for developer-controlled evaluation workflows, and no evidence of exploitable capabilities (like writing back to the environment or exfiltrating data) was found.- [REMOTE_CODE_EXECUTION]: There are no patterns of remote code execution. The Python scripts included (scripts/generate_example_specs.pyandassets/examples/plugin_sdk_examples.py) are local utility scripts for generating spec files and demonstrating SDK usage. No external scripts are downloaded or piped to a shell.
Audit Metadata