add-benchmark
Installation
SKILL.md
Add SWE Benchmark
Add a new benchmark task to the bench-swe pipeline from a real GitHub bug-fix. The human provides the GitHub issue or PR URL; the agent handles extraction, validation, and file creation.
Arguments
- url (required): GitHub issue or PR URL (e.g.
https://github.com/gorilla/mux/issues/534orhttps://github.com/gorilla/mux/pull/585) - language (required): One of: go, python, typescript, javascript, rust, ruby, java, c, cpp, php, csharp
Repository selection criteria
Good benchmark repos are focused libraries with a clear bug — not large applications. Before submitting a URL, prefer repos that are: