evaluating-llms-harness
Pass
Audited by Gen Agent Trust Hub on Sep 16, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONDYNAMIC_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
- [EXTERNAL_DOWNLOADS]: The skill references downloads for models, datasets, and container images from well-known and trusted sources, including HuggingFace (e.g., meta-llama, mistralai, microsoft), GitHub (EleutherAI), and the GitHub Container Registry. These references are standard for model benchmarking and utilize official repositories.
- [COMMAND_EXECUTION]: The instructions provide examples of using shell scripts and the
os.systemfunction within Python training loops to automate benchmarks. These examples are clearly intended for monitoring training progress and are scoped to the user's local development environment. - [DYNAMIC_EXECUTION]: The skill documents the use of the
--allow_code_executionflag, which is a core requirement for evaluating models on functional correctness benchmarks like HumanEval. It also provides boilerplate for Python utilities to execute and score model-generated code, which is the primary purpose of a code-benchmarking harness. - [INDIRECT_PROMPT_INJECTION]: As an evaluation harness, the skill naturally processes untrusted data in the form of academic datasets and model outputs. This ingestion surface is fundamental to the skill's utility and follows industry-standard benchmarking protocols for academic and research purposes.
Audit Metadata