The Agent Skills Directory

[COMMAND_EXECUTION]: The documentation describes using the lm_eval CLI and provides Python examples using os.system to automate evaluation workflows. These are standard practices for the benchmarking tool and do not involve unauthorized execution.
[EXTERNAL_DOWNLOADS]: The skill recommends installing established packages like lm-eval, transformers, and vllm from public registries. References to datasets and model weights target well-known repositories like HuggingFace and official project pages.
[REMOTE_CODE_EXECUTION]: The instructions explicitly mention the --allow_code_execution flag, which is a required feature for evaluating model-generated code in benchmarks like HumanEval. This is documented as a necessary step for the specific task.
[CREDENTIALS_UNSAFE]: The skill provides instructions for configuring API keys via environment variables using placeholder values, adhering to standard security practices for managing secrets.

evaluating-llms-harness