The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The skill provides instructions to download and install the core xerj binary from the vendor's official website (xerj.org) using a shell pipe. It also fetches corpus manifests and repository data from the vendor's public GitHub repository and other well-known services.
[COMMAND_EXECUTION]: The skill relies on executing system commands, including git for repository management, the xerj binary for indexing and searching, and the claude CLI for agent interaction. These are used to automate the reference-coding workflow and the evaluation of model performance.
[INDIRECT_PROMPT_INJECTION]: The benchmarking harness in swebench/gen_predictions.py possesses an attack surface for indirect prompt injection as it ingests untrusted problem descriptions from external datasets.
Ingestion points: Untrusted data enters the agent context via the load_dataset function in swebench/gen_predictions.py, specifically the problem_statement field.
Boundary markers: The script interpolates these external strings directly into the prompt in the run_agent function without the use of protective delimiters or specific instructions to ignore embedded commands.
Capability inventory: The script has access to subprocess.run to execute git, xerj, and claude commands, as well as file system operations for temporary directory management.
Sanitization: No input sanitization or validation is performed on the ingested problem statements before they are passed to the agent.
[PRIVILEGE_ESCALATION]: The automated benchmarking script (gen_predictions.py) invokes the Claude CLI using the --permission-mode bypassPermissions flag. This configuration suppresses user confirmation prompts for the agent's tool executions, facilitating unattended automated benchmarking runs while reducing manual oversight of the agent's actions within the benchmarking environment.

xerj-code