ragcheck
Installation
SKILL.md
ragcheck
Offline and deterministic: no network, no model calls. Inputs are JSON arrays (not JSONL); schemas in references/REFERENCE.md, samples in examples/.
Get the binary
brew install itamaker/tap/ragcheck, a release binary from github.com/itamaker/ragcheck-skill, or go build -o ragcheck . in a clone of that repo.
Bare ragcheck opens an interactive TUI that hangs an agent shell; always pass a subcommand. Builds after v0.3.0 answer ragcheck -h and ragcheck --version (exit 0), and ragcheck score -h lists the flags. With v0.3.0 these exit 2, which is expected: run ragcheck x (an invalid subcommand) for usage.
Run
ragcheck score -qrels qrels.json -run run.json -k 5 [-json] # retrieval metrics
ragcheck judge -input judge.json [-json] # lexical answer check; per-query hints need -json
Compare retriever configs by running score once per -run file with the same -qrels and -k, then diffing the numbers.