The Agent Skills Directory

[DYNAMIC_EXECUTION]: The skill relies on JSON specification files located in assets/specs/ that contain base64-encoded cloudpickle blobs. These blobs are deserialized at runtime to execute evaluation logic. Because pickle and cloudpickle can execute arbitrary code during the deserialization process, this represents a significant security risk if a user or an automated process modifies these local files with malicious intent.
[COMMAND_EXECUTION]: The skill provides numerous instructions for shell command execution, including the use of the nemo CLI and local Python scripts. Notably, scripts/generate_example_specs.py uses os.execvpe to replace the current process and re-run itself with a specific environment configuration (PYTHONHASHSEED), which is a form of dynamic process management.
[INDIRECT_PROMPT_INJECTION]: The LLM-judge configuration in assets/specs/llm_as_judge.json contains a prompt template that directly interpolates untrusted data from the dataset field (e.g., {{item.input}}). This creates an attack surface for indirect prompt injection, where malicious content in the dataset could influence the behavior of the judge model. The skill lacks strong boundary markers or explicit sanitization instructions to mitigate this risk.
[DATA_EXFILTRATION]: While the skill does not show signs of malicious exfiltration, it is designed to handle sensitive credentials like NVIDIA_API_KEY through environment variables and platform secrets. The instructions in references/api-auth.md correctly advise on secure secret management, but the overall capability to read and process these secrets across local and remote environments requires a high degree of trust in the orchestration scripts.

nemo-evaluator-plugin