The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The documentation provides instructions to install Python packages directly from GitHub repository commits during the setup phase of certain benchmarks.
File references/benchmarks/perspective_gap_prompt_writing.md suggests installing from github.com/WhymustIhaveaname/PerspectiveGap.git.
File references/benchmarks/claw_eval.md suggests installing from github.com/claw-eval/claw-eval.git.
[REMOTE_CODE_EXECUTION]: The framework is designed to execute code generated by models or provided within datasets for evaluation tasks.
Many benchmarks described, such as bigcodebench, humaneval, and swe_bench, involve the execution of Python or shell scripts generated by the agent or model being tested.
The skill mitigates this risk by recommending and providing configurations for isolated Docker-based sandbox environments (ms-enclave).
[CREDENTIALS_UNSAFE]: Example code for specific benchmark tasks contains hardcoded credentials for public testing services.
File references/benchmarks/bigcodebench_hard.md includes an FTP password (rNrKYTX9g7z3RgJRmxWuGHbeu) for the public site ftp.dlptest.com. This is part of the task definition but matches credential patterns.
[PROMPT_INJECTION]: The skill facilitates the processing of large datasets sourced from the internet, which creates an indirect prompt injection surface.
Ingestion points include GitHub issues in swe_bench and various web-scraped documents in retrieval benchmarks.
The documentation provides boundary markers and sandbox configurations to prevent model-generated content from escaping the execution environment.

evalscope-docs