The Agent Skills Directory

[DYNAMIC_EXECUTION]: The skill implements an adapter system that allows for the dynamic discovery and loading of custom Python interceptors from specified modules or local directories using the discovery.modules and discovery.dirs configuration fields in references/adapter-system.md.- [COMMAND_EXECUTION]: The benchmarking launcher assembles and executes shell commands based on user-defined templates in Framework Definition Files (FDFs). These templates, such as python -m my_custom_eval.run --model-id {model_id} --task {task} --output-dir {output_dir}, provide a mechanism for arbitrary command execution as documented in references/custom-benchmarks.md.- [REMOTE_CODE_EXECUTION]: The Slurm execution backend establishes SSH connections to remote HPC head nodes to perform job orchestration tasks, including the execution of sbatch, squeue, and sacct commands on the remote system as described in references/execution-backends.md.- [CREDENTIALS_UNSAFE]: The skill manages sensitive credentials including API keys (NGC, HF, OpenAI) and SSH private keys. Specifically, the Slurm executor relies on the ssh_key_path (defaulting to ~/.ssh/id_rsa) to authenticate remote operations on cluster infrastructure.- [INDIRECT_PROMPT_INJECTION]: The skill processes untrusted model responses from external endpoints across numerous benchmarks, creating a potential attack surface for indirect prompt injection.
Ingestion points: Model outputs are ingested by the core library and processed by interceptors (e.g., the reasoning interceptor) and framework-specific output parsers.
Boundary markers: The documentation does not specify explicit boundary markers or instructions to ignore embedded commands in model responses.
Capability inventory: The skill possesses capabilities for file system writes (result artifacts), network communication (model APIs and progress tracking), and command execution (benchmark harnesses).
Sanitization: While the reasoning interceptor can strip specific tokens, there is no detailed evidence of generalized sanitization or validation of model-generated content before it is parsed into metrics.- [EXTERNAL_DOWNLOADS]: The skill fetches containerized benchmark environments from NVIDIA's official container registry (nvcr.io/nvidia/eval-factory/) and installs dependencies from public package registries.

nemo-evaluator-sdk