The Agent Skills Directory

[DYNAMIC_EXECUTION]: The skill implements a discovery mechanism for custom interceptors, enabling the evaluation engine to load and execute code from arbitrary local Python modules or directory paths specified in the configuration files.
[CREDENTIALS_UNSAFE]: To support the Slurm execution backend, the system requires access to the user's private SSH key (e.g., ~/.ssh/id_rsa) to facilitate automated connectivity and job submission to remote HPC clusters.
[COMMAND_EXECUTION]: The framework relies on a launcher tool that executes shell commands for container orchestration, remote job management via SSH/sbatch, and result processing, representing the primary operational mode of the toolkit.
[INDIRECT_PROMPT_INJECTION]: The skill processes untrusted model responses through multiple interceptor stages and output parsers. An attacker-controlled model could potentially provide outputs designed to exploit these processing layers, though impact is limited by the containerized execution environment.
Ingestion points: Model responses collected from target API endpoints during benchmarking tasks.
Boundary markers: Documentation does not specify mandatory delimiters or specific instructions for the agent to ignore instructions embedded within the model's performance data.
Capability inventory: The launcher environment possesses capabilities for subprocess execution (Docker/Slurm), file system modification (results and artifacts), and network operations (exporting results to MLflow/W&B).
Sanitization: Results are processed by individual framework output parsers which transform raw API responses into metric schemas.
[EXTERNAL_DOWNLOADS]: The skill fetches container images from NVIDIA's official registry (nvcr.io) and references resources from the NeMo Evaluator GitHub organization, which are established and well-known services for this technology stack.

nemo-evaluator-sdk