evaluating-code-models
Pass
Audited by Gen Agent Trust Hub on Jun 29, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONREMOTE_CODE_EXECUTION
Full Analysis
- [EXTERNAL_DOWNLOADS]: The skill facilitates downloading the BigCode Evaluation Harness repository directly from its official GitHub source (bigcode-project/bigcode-evaluation-harness.git). It also installs standard machine learning dependencies from official registries like PyPI and the PyTorch index.
- [COMMAND_EXECUTION]: The skill instructions involve executing model-generated code via the
--allow_code_executionflag. While this involves running untrusted code, the documentation explicitly identifies the risks and provides remediation guidance, such as using Docker containers for isolation. - [REMOTE_CODE_EXECUTION]: Some configurations utilize the
--trust_remote_codeflag when loading models from Hugging Face. This allows the execution of model-specific scripts during the loading process, which is a standard procedure for many specialized architectures from well-known model providers. - [DATA_EXFILTRATION]: The skill includes procedures for handling Hugging Face authentication tokens (
--use_auth_token). These are standard requirements for accessing gated research models and datasets, and the skill provides instructions for secure login using official CLI tools.
Audit Metadata