The Agent Skills Directory

[CREDENTIALS_UNSAFE]: The skill configuration refers to sensitive file paths for SSH authentication. The SlurmExecutor configuration in references/execution-backends.md includes ssh_key_path: ~/.ssh/id_rsa as an example for connecting to remote clusters.\n- [PRIVILEGE_ESCALATION]: The skill documentation provides instructions for escalating user privileges. In references/execution-backends.md, the troubleshooting section suggests executing sudo usermod -aG docker $USER to grant the user Docker group permissions.\n- [DYNAMIC_EXECUTION]: The system allows for dynamic loading and execution of Python code from specified paths. As described in references/adapter-system.md, the DiscoveryConfig enables the discovery of custom interceptors from modules (discovery.modules) or directories (discovery.dirs).\n- [COMMAND_EXECUTION]: The skill executes various system commands through its execution backends. The launcher uses subprocess to call commands like docker, sbatch, squeue, sacct, and ssh as part of its core evaluation workflows.\n- [INDIRECT_PROMPT_INJECTION]: The skill has an attack surface for indirect prompt injection as it processes external datasets and model responses. \n
Ingestion points: Benchmark datasets loaded via dataset_dir and responses received from api_endpoint (found in references/configuration.md and references/custom-benchmarks.md).\n
Boundary markers: None identified in the provided configuration examples for isolating external input.\n
Capability inventory: The skill possesses extensive capabilities, including executing subprocesses for Docker and Slurm, writing to the filesystem, and performing network operations (API requests, progress tracking).\n
Sanitization: No evidence of input sanitization or validation of model outputs was found in the configuration or reference documentation.\n- [DATA_EXFILTRATION]: The skill performs network operations to user-configurable endpoints. The progress_tracking_url in references/adapter-system.md allows sending evaluation progress to an external URL, and the SlurmExecutor performs SSH connections to remote hostnames.

nemo-evaluator-sdk