evaluating-code-models

MEDIUM W011: Third-party content exposure detected (indirect prompt injection risk).

• Third-party content exposure detected (high risk: 0.75). SKILL.md describes running evaluation tasks (e.g., HumanEval/MBPP/MultiPL-E) whose prompts/tests are sourced from external datasets at runtime (HuggingFace datasets / public benchmark content), which are outsider-authored free text that the harness will feed into the model context.

MEDIUM W012: Unverifiable external dependency detected (runtime URL that controls agent).

• Potentially malicious external URL detected (high risk: 0.90). The skill contains commands that fetch and run external code at runtime—for example pulling and running the Docker image ghcr.io/bigcode-project/evaluation-harness-multiple (docker pull/run) and cloning then installing the repository via git clone https://github.com/bigcode-project/bigcode-evaluation-harness.git followed by pip install -e ., which fetch and execute remote code.