The Agent Skills Directory

[REMOTE_CODE_EXECUTION]: The skill documents the use of the --allow_code_execution flag for the HumanEval benchmark. This feature executes model-generated code to verify functional correctness, which is the standard methodology for this specific academic benchmark. The skill correctly identifies this as a requirement for that task.
[DYNAMIC_EXECUTION]: Custom tasks are defined using YAML configurations that utilize !function tags to invoke Python logic from local utils.py files. This is a core architectural feature of the lm-eval-harness task system used for extending its benchmarking capabilities to proprietary datasets.
[COMMAND_EXECUTION]: The skill provides automation examples using os.system and subprocess to manage benchmarking workflows and execute the CLI tool programmatically within training loops. These are standard practices in machine learning research automation.
[DATA_EXFILTRATION]: Documentation for local API evaluation mentions the verify_certificate=false option. This is explicitly labeled for development use only, allowing testing against local inference servers (like vLLM or Ollama) that may not have valid SSL certificates configured.

evaluating-llms-harness