sst-llm-judge-ranker

Installation
SKILL.md

LLM-judge ranker

Order N artifacts by quality on a stated dimension. The comparator is "the LLM judges A vs B against this goal," done pairwise.

Project contract

  • Output dir: <project>/data/sst-llm-judge-ranker/<run-id>/ where <run-id> is either user-supplied or auto-generated as <utc>_<short-uuid>. Inside:
    • metadata.jsonl — append-only log of every artifact registered.
    • ranking_state/<artifact-set>.ranklist.json — the current ranked list (just IDs, ordered best→worst).
  • Tools required: harness's Read (for candidate files). No web access; ranking is pure analysis.
  • Input shapes:
    • Direct compare: two file paths + an optional goal → returns winner + rationale.
    • Batch rank: a directory of candidate files + a goal → produces an initial ranklist.
    • Incremental insert: a new candidate path + the existing ranklist file → places the new candidate at its correct position with binary-search comparisons.

Operating principles

Installs
1
GitHub Stars
3
First Seen
Jun 16, 2026
sst-llm-judge-ranker — aibot88/sec_skill_store