The Agent Skills Directory

[REMOTE_CODE_EXECUTION]:
Evidence: Evaluation specification files in assets/specs/ (specifically llm_as_judge.json, exact_match_metric.json, and exact_match_benchmark.json) store metric configurations as cloudpickle payloads in the blob field.
Risk: Deserializing these assets via the Evaluator CLI or SDK allows for arbitrary Python code execution.
[COMMAND_EXECUTION]:
Evidence: The utility script scripts/generate_example_specs.py invokes os.execvpe to replace the current process with a new Python instance to enforce a deterministic PYTHONHASHSEED environment variable.
[PROMPT_INJECTION]:
Ingestion points: The dataset field and metric prompt template in assets/specs/llm_as_judge.json (found in SKILL.md and referenced spec files) ingest untrusted input data (item.input) and model output (sample.output_text) into evaluation prompts.
Boundary markers: Absent. The prompt template does not use delimiters to isolate untrusted data from instructions.
Capability inventory: The skill executes evaluation workflows including LLM-based judging and job submission via nemo evaluator and nemo jobs commands.
Sanitization: No evidence of escaping or sanitization of the interpolated variables was found, making the process vulnerable to indirect prompt injection where malicious content in the dataset could influence the judge's scoring or behavior.

nemo-evaluator-plugin