The Agent Skills Directory

[INDIRECT_PROMPT_INJECTION]: The skill processes untrusted outputs from language models to calculate benchmarks, creating an attack surface where malicious model output could target the evaluator or the agent environment.
Ingestion points: Model predictions across 60+ benchmarks (e.g., MMLU, GSM8K) and custom datasets described in SKILL.md and references/custom-tasks.md.
Boundary markers: The instructions use standardized prompt templates, but the snippets do not show explicit sanitization or strict delimiters to prevent instruction leakage from model outputs.
Capability inventory: The skill documentation suggests using os.system for automation (SKILL.md) and subprocess for code execution during benchmarks (references/custom-tasks.md).
Sanitization: Absent in the provided workflow examples, as the focus is on metric extraction.
[DYNAMIC_EXECUTION]: The skill facilitates the execution of generated code at runtime to verify functional correctness for benchmarks like HumanEval.
The documentation explicitly notes that the --allow_code_execution flag is required for these tasks, placing the security choice with the user.
Example utilities in references/custom-tasks.md demonstrate the use of subprocess to run code blocks against test cases for code evaluation.
[COMMAND_EXECUTION]: The skill provides scripts and Python snippets that use os.system and shell commands to orchestrate training checkpoints and aggregate multi-model results in SKILL.md.
[EXTERNAL_DOWNLOADS]: The skill references and installs well-known packages from established repositories and services, including lm-eval from EleutherAI, transformers from HuggingFace, and vllm. These are recognized as standard, reputable sources in the AI community.

evaluating-llms-harness