The Agent Skills Directory

[COMMAND_EXECUTION]: The skill provides detailed implementation patterns for evaluating model output via subprocess execution. This includes the use of subprocess.run and Ray remote tasks to compile and run code. The documentation provides best practices for this pattern, such as using tempfile.TemporaryDirectory for isolation and asyncio.Semaphore to manage concurrency levels.
[EXTERNAL_DOWNLOADS]: The instructions describe an 'Auto-install external tools' pattern. This allows benchmarks to automatically set up required compilers or runtimes on the host system using shell scripts (e.g., install_tool.sh) or package managers like brew during the server's initialization phase.
[CREDENTIALS_UNSAFE]: The documentation guides users to store sensitive GitLab API tokens and MLflow tracking URIs within an env.yaml file. This is presented as a standard method for authenticating with the dataset registry to upload or download training and validation data.

add-benchmark