The Agent Skills Directory

[COMMAND_EXECUTION]: The skill provides instructions for the agent to use the nemo-evaluator-launcher CLI, which executes shell commands, manages Docker containers, and submits batch jobs to Slurm HPC clusters. It also supports Framework Definition Files (FDFs) that utilize command templates (e.g., python -m my_custom_eval.run ...) executed via subprocesses.
[DYNAMIC_EXECUTION]: The framework's adapter system includes a DiscoveryConfig feature which allows the application to dynamically discover and load Python modules or interceptor components from specified local directories or module paths at runtime.
[EXTERNAL_DOWNLOADS]: The skill facilitates the installation of the nemo-evaluator-launcher Python package via standard package managers and pulling specialized evaluation containers from the official NVIDIA GPU Cloud (NGC) registry (nvcr.io).
[REMOTE_CODE_EXECUTION]: The skill orchestrates the downloading and execution of remote Docker container images for task-specific evaluations. These images are sourced from trusted NVIDIA repositories.
[INDIRECT_PROMPT_INJECTION]: The skill architecture is designed to ingest and process external data, including YAML configuration files, task definitions, and raw API responses from the models under evaluation.
Ingestion points: Configuration files (config.yaml), framework definitions, mapping files (mapping.toml), and runtime model responses handled by the interceptor pipeline.
Boundary markers: The documentation does not specify boundary markers or explicit isolation protocols to prevent instructions inside external configurations or model responses from influencing the launcher's logic.
Capability inventory: The skill has access to shell execution, container management, HPC job submission tools, and network communication for external API calls.
Sanitization: There are no documented sanitization routines for the interpolation of configuration values or task names into command templates.

nemo-evaluator-sdk