hugging-face-community-evals

Pass

Audited by Gen Agent Trust Hub on Oct 26, 2026

Risk Level: SAFEREMOTE_CODE_EXECUTIONCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTION
Full Analysis
  • [REMOTE_CODE_EXECUTION]: The scripts scripts/inspect_vllm_uv.py and scripts/lighteval_vllm_uv.py include support for the --trust-remote-code argument. This parameter allows the underlying Hugging Face Transformers and vLLM libraries to download and execute arbitrary Python code contained within a model repository on the Hugging Face Hub. This is a standard mechanism for handling custom model architectures but introduces a risk if a model repository is compromised or malicious.
  • [COMMAND_EXECUTION]: The skill uses subprocess.run across all scripts in the scripts/ directory to invoke the inspect and lighteval CLI tools. While the commands are constructed using list arguments (mitigating shell injection vulnerabilities), the skill acts as a bridge for executing local system commands based on user-provided arguments such as model IDs and task names.
  • [DYNAMIC_EXECUTION]: The --trust-remote-code functionality enables dynamic loading and execution of Python code from remote sources at runtime. This allows code not present in the skill's distribution to be executed on the host system during model loading.
  • [INDIRECT_PROMPT_INJECTION]: The skill processes untrusted data in the form of models and benchmarks retrieved from the Hugging Face Hub. This creates a surface for indirect prompt injection where malicious instructions could be embedded in model files or benchmark data.
  • Ingestion points: Model repositories and benchmark datasets from the Hugging Face Hub (via scripts/ entry points).
  • Boundary markers: None identified; inputs are passed directly to evaluation frameworks.
  • Capability inventory: Local shell command execution via subprocess.run in scripts/inspect_eval_uv.py, scripts/inspect_vllm_uv.py, and scripts/lighteval_vllm_uv.py.
  • Sanitization: None; the skill relies on the security boundaries of the underlying inspect-ai and lighteval frameworks.
Audit Metadata
Risk Level
SAFE
Analyzed
Oct 26, 2026, 05:16 PM