The Agent Skills Directory

[DYNAMIC_EXECUTION]: The skill dynamically loads and executes Python code from the repository being analyzed.
Evidence: scripts/providers/harbor/_judge.py uses import_class to load agent code from paths defined in repository configurations.
Evidence: scripts/providers/gym/_judge.py generates a shell/Python wrapper (_LAUNCHER) that uses importlib.util.module_from_spec and spec.loader.exec_module to execute a verifier.py script provided by the repository.
[COMMAND_EXECUTION]: The skill executes various system commands to perform its tasks.
Evidence: scripts/providers/harbor/_solve.py executes harbor run, which triggers the execution of repository-supplied solution scripts (solve.sh).
Evidence: scripts/providers/gym/_judge.py and scripts/providers/gym/_solve.py execute the gym CLI tool for manifest validation and testing.
Evidence: scripts/_other_evals.py executes git ls-files to scan for evaluation-like code patterns.
[INDIRECT_PROMPT_INJECTION]: The skill processes untrusted data from the analyzed repository to guide its execution flow.
Ingestion points: scripts/providers/harbor/_explore.py and scripts/providers/gym/_explore.py read YAML, JSON, and JSONL configuration files from the target repository.
Boundary markers: The skill uses yaml.safe_load (or falls back to a regex key scan) to parse configuration files.
Capability inventory: The skill can execute subprocesses (harbor, gym, git), read/write files in scratch directories, and dynamically import Python modules.
Sanitization: Uses yaml.safe_load for structured data and applies basic regex-based name sanitization (_UNSAFE_NAME) in scripts/providers/gym/_judge.py for task IDs.

eval-author-discover