evaluation

Warn

Audited by Socket on Aug 29, 2026

5 alerts found:

Anomalyx4Security
AnomalyLOW
SKILL.md

SUSPICIOUS. The skill’s core purpose is legitimate and largely aligned with NVIDIA’s official evaluator tooling, with install guidance mostly pointing to same-org or official registries. However, the operational footprint is broad for an AI skill: it reads/sources sensitive env files, forwards multiple credentials into external runtimes, executes bundled helper scripts, launches remote cluster jobs over SSH, and can push outputs to MLflow and AWS/Harbor-backed paths. These actions are proportionate to large-scale evaluation workflows, but they still create medium risk because the agent gains meaningful access to secrets and remote execution surfaces. I do not see strong evidence of malicious intent or obvious credential-harvesting proxies, but the credential handling and remote execution scope exceed low-risk documentation-only skills.

Confidence: 89%Severity: 58%
SecurityMEDIUM
scripts/nel-next.sh

No direct evidence of overt malware (backdoor, credential theft, reverse shell) appears in the provided fragment. However, the module has significant security exposure: it uses eval "$cfgvars" on configuration-derived content, which can enable arbitrary local command execution if inputs are not strictly trusted/sanitized. It also performs remote file collection and then exports evaluation bundles to MLflow using a configurable tracking_uri (with an external default), creating a strong data egress path. Treat this as a high-impact orchestration component intended for trusted environments only, and prioritize removing/replacing eval and enforcing strict validation of config/environment values (especially tracking_uri and fields used to build cfgvars).

Confidence: 60%Severity: 74%
AnomalyLOW
recipes/examples/gym/example_gdpval.yaml

No clear evidence of intentional malware or covert data theft exists in this fragment. The main concerns are supply-chain integrity risks: unpinned apt installation from a PPA, conditional git fetch/checkout of external code, and runtime pip installation from requirements files. Secrets are handled via env vars with an explicit attempt to avoid leaking them in logs (set +x). Overall, treat as medium supply-chain risk due to dynamic dependency acquisition and networked execution, but likelihood of direct sabotage/malware is not supported by explicit malicious indicators in the provided code.

Confidence: 66%Severity: 52%
AnomalyLOW
recipes/examples/gym/example_mrcr.yaml

No clear evidence of intentional malware/backdoor behavior is present in this YAML fragment. The main security concern is supply-chain execution risk: it runs a curl|sh installer for uv, fetches and checks out a GitHub repo at runtime (pinned commit, which reduces but does not eliminate risk), and installs/modifies Python dependencies before executing benchmark commands. It also enables logging (MLflow + request/response logging), which could inadvertently record sensitive data depending on tool behavior. Overall: likely legitimate CI/evaluation bootstrap code, but it warrants scrutiny because it performs runtime code and dependency fetching/execution.

Confidence: 66%Severity: 62%
AnomalyLOW
recipes/examples/example_eval_next.yaml

No explicit malicious code/backdoor indicators are visible in this fragment because it is declarative configuration. However, it has notable security and supply-chain risk drivers: it injects AWS access keys and a HuggingFace token into a container runtime determined by a variable image reference, and it enables MLflow export with log copying that may upload captured request bodies or other sensitive artifacts depending on upstream logging/exclusion completeness. Main recommended checks are container image integrity/provenance (pin/digest), least-privilege credential scoping, and strict control of what artifacts/logs are exported to MLflow.

Confidence: 41%Severity: 52%
Audit Metadata
Analyzed At
Aug 29, 2026, 04:17 PM
Package URL
pkg:socket/skills-sh/nvidia%2Fmodel-optimizer%2Fevaluation%2F@8ae91da26c378f7e2c82b26b8d9a2d50a17ccc63ec264f8fb25701338905d172
Security Audit — socket — evaluation