evaluating-code-models
Pass
Audited by Gen Agent Trust Hub on Sep 16, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSINDIRECT_PROMPT_INJECTIONREMOTE_CODE_EXECUTIONPRIVILEGE_ESCALATIONCOMMAND_EXECUTION
Full Analysis
- [EXTERNAL_DOWNLOADS]: Fetches the BigCode evaluation harness repository from GitHub. The repository belongs to a well-known research project and organization.
- [INDIRECT_PROMPT_INJECTION]: The skill processes and executes untrusted code generated by AI models during evaluation. Ingestion points: Model outputs from benchmarking tasks such as HumanEval and MBPP are ingested by the evaluation script. Boundary markers: No explicit markers are used for raw code execution; however, the skill documents the use of Docker for safe isolation. Capability inventory: The evaluation tool can execute arbitrary code on the host machine when the allow_code_execution flag is enabled. Sanitization: The skill recommends running evaluations within isolated Docker containers to mitigate the risk of host system compromise.
- [REMOTE_CODE_EXECUTION]: Instructions specify the use of the trust_remote_code flag for certain models, which allows the execution of custom Python code defined within Hugging Face model repositories.
- [PRIVILEGE_ESCALATION]: Troubleshooting documentation includes commands that use sudo to build Docker images locally.
- [COMMAND_EXECUTION]: The skill utilizes shell commands to install dependencies, clone repositories, and run benchmarking scripts.
Audit Metadata