The Agent Skills Directory

[REMOTE_CODE_EXECUTION]: The evaluation specification files (assets/specs/exact_match_benchmark.json, assets/specs/exact_match_metric.json, and assets/specs/llm_as_judge.json) contain serialized Python objects within the payload.blob fields using the cloudpickle format. Deserializing such content is a form of dynamic execution that can lead to arbitrary code execution within the environment where the evaluation is performed.
[COMMAND_EXECUTION]: The skill instructions involve executing shell commands through the nemo CLI and running local Python scripts, such as scripts/generate_example_specs.py, which may perform operations on the host system.
[PROMPT_INJECTION]: The skill processes external datasets that are interpolated into prompts (e.g., {{item.input}} in llm_as_judge.json), creating a surface for indirect prompt injection.
Ingestion points: Dataset fields in specification files within assets/specs/.
Boundary markers: Structured messages array with role definitions (system, user).
Capability inventory: Execution of shell commands via the nemo CLI.
Sanitization: No evidence of input validation or sanitization for dataset content.

nemo-evaluator-plugin