sst-llm-judge-ranker
Installation
SKILL.md
LLM-judge ranker
Order N artifacts by quality on a stated dimension. The comparator is "the LLM judges A vs B against this goal," done pairwise.
Project contract
- Output dir:
<project>/data/sst-llm-judge-ranker/<run-id>/where<run-id>is either user-supplied or auto-generated as<utc>_<short-uuid>. Inside:metadata.jsonl— append-only log of every artifact registered.ranking_state/<artifact-set>.ranklist.json— the current ranked list (just IDs, ordered best→worst).
- Tools required: harness's
Read(for candidate files). No web access; ranking is pure analysis. - Input shapes:
- Direct compare: two file paths + an optional goal → returns winner + rationale.
- Batch rank: a directory of candidate files + a goal → produces an initial ranklist.
- Incremental insert: a new candidate path + the existing ranklist file → places the new candidate at its correct position with binary-search comparisons.