The Agent Skills Directory

[COMMAND_EXECUTION]: Several scripts, including inspect_eval_uv.py, inspect_vllm_uv.py, lighteval_vllm_uv.py, run_eval_job.py, and run_vllm_eval_job.py, utilize the subprocess module to run CLI tools like hf, inspect, and lighteval. These shell commands are constructed using arguments provided by the user, such as model identifiers and task names.
[REMOTE_CODE_EXECUTION]: The scripts inspect_vllm_uv.py, lighteval_vllm_uv.py, and run_vllm_eval_job.py provide a --trust-remote-code parameter. This flag allows the underlying Hugging Face Transformers and vLLM libraries to download and run arbitrary Python scripts hosted within model repositories, which represents a significant risk if the model source is untrusted.
[EXTERNAL_DOWNLOADS]: The skill fetches model metadata, README files, and evaluation data from huggingface.co and artificialanalysis.ai. It also downloads model weights and associated repository files during the evaluation process.
[PROMPT_INJECTION]: The evaluation_manager.py script parses and extracts data from model READMEs. This creates an indirect prompt injection surface where a malicious repository owner could embed instructions within markdown tables or comments to influence the behavior of the agent or the parsing logic.
Ingestion points: scripts/evaluation_manager.py reads README content via the Hugging Face API.
Boundary markers: None specific; content is processed as raw markdown.
Capability inventory: The skill can write to Hugging Face repositories, execute shell commands, and make network requests.
Sanitization: Basic cleaning is performed on strings, and numeric values are validated through float conversion.

hugging-face-evaluation