hugging-face-community-evals
Pass
Audited by Gen Agent Trust Hub on Oct 26, 2026
Risk Level: SAFEREMOTE_CODE_EXECUTIONCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTION
Full Analysis
- [REMOTE_CODE_EXECUTION]: The scripts
scripts/inspect_vllm_uv.pyandscripts/lighteval_vllm_uv.pyinclude support for the--trust-remote-codeargument. This parameter allows the underlying Hugging Face Transformers and vLLM libraries to download and execute arbitrary Python code contained within a model repository on the Hugging Face Hub. This is a standard mechanism for handling custom model architectures but introduces a risk if a model repository is compromised or malicious. - [COMMAND_EXECUTION]: The skill uses
subprocess.runacross all scripts in thescripts/directory to invoke theinspectandlightevalCLI tools. While the commands are constructed using list arguments (mitigating shell injection vulnerabilities), the skill acts as a bridge for executing local system commands based on user-provided arguments such as model IDs and task names. - [DYNAMIC_EXECUTION]: The
--trust-remote-codefunctionality enables dynamic loading and execution of Python code from remote sources at runtime. This allows code not present in the skill's distribution to be executed on the host system during model loading. - [INDIRECT_PROMPT_INJECTION]: The skill processes untrusted data in the form of models and benchmarks retrieved from the Hugging Face Hub. This creates a surface for indirect prompt injection where malicious instructions could be embedded in model files or benchmark data.
- Ingestion points: Model repositories and benchmark datasets from the Hugging Face Hub (via
scripts/entry points). - Boundary markers: None identified; inputs are passed directly to evaluation frameworks.
- Capability inventory: Local shell command execution via
subprocess.runinscripts/inspect_eval_uv.py,scripts/inspect_vllm_uv.py, andscripts/lighteval_vllm_uv.py. - Sanitization: None; the skill relies on the security boundaries of the underlying
inspect-aiandlightevalframeworks.
Audit Metadata