evaluation
Fail
Audited by Gen Agent Trust Hub on May 18, 2026
Risk Level: HIGHCOMMAND_EXECUTIONREMOTE_CODE_EXECUTIONDATA_EXFILTRATIONCREDENTIALS_UNSAFEPROMPT_INJECTION
Full Analysis
- [DATA_EXFILTRATION]: The skill instructions in Step 7.5 direct the agent to execute shell commands via SSH to read the
~/.config/enroot/.credentialsfile on a remote host. This file typically contains sensitive authentication credentials for container registries. - [REMOTE_CODE_EXECUTION]: The skill documentation in
references/model-card-research.mdandSKILL.mdencourages the use of thedeployment.pre_cmdfield to download and execute external scripts (e.g., usingcurlandpip install). The skill further instructs the agent to setNEMO_EVALUATOR_TRUST_PRE_CMD=1, which explicitly enables the execution of these arbitrary commands by the underlying evaluation tool. - [COMMAND_EXECUTION]: The skill performs shell execution of various commands including
nel,cat, andssh. These commands incorporate parameters and arguments dynamically derived from untrusted external data sources such as HuggingFace model cards and checkpoint metadata files. - [PROMPT_INJECTION]: The skill has a significant surface for indirect prompt injection. It is instructed to use
WebSearchto research model cards on HuggingFace and to read local or remote checkpoint files (config.json,hf_quant_config.json) to automatically determine deployment configurations. Maliciously crafted data in these external files could trick the agent into generating dangerous command-line arguments or maliciouspre_cmdpayloads. - [CREDENTIALS_UNSAFE]: The skill targets sensitive credential files and environment variables. While it suggests using
.envfiles for secret management (a safe practice), the automated inspection of~/.config/enroot/.credentialsconstitutes an unsafe credential access pattern.
Recommendations
- AI detected serious security threats
Audit Metadata