media-judge
Media judge
Measure, don't grade. A judgment here is a time, a number and a picture. "Looks great", "8/10" and "feels premium" are not findings, and neither is any adjective with no measurement behind it.
Code finds the moment and a fresh pair of eyes reads only that moment, in that order, because the other way round fails: 12 vision models scored at most 54.1% on spotting temporal glitches; Claude Opus caught 17.5% of them, Gemini 3 Flash said "glitch" to about 87% of clean clips (TempGlitch, gameplay clips). Uniform sampling at 0.5 fps caught 11.81% of faults under 0.2 s (SynFlash), and GPT-4 preferred its own output 0.705 and 0.912 of the time on tasks where people saw equal quality (Panickssery et al.). So:
- Facts:
judge.pyreads every frame and every sample. It does not sample. - Eyes: a fresh agent opens only the evidence and answers narrow questions.
- Verdict: counted defects. Ranking follows from the counts, never from a feeling.
What this has been tested on. Planted defects in footage it never saw while being tuned: 18 of 22 caught, 6 of 8 clean controls (25 of 25 and 10 of 10 on the footage it was tuned on). In a blind test on 19 anonymous clips, an agent with this skill found 12 of 14 planted defects, with no false claims on clean clips, using 116k tokens. An agent given only the failure list found 11 of 14, using 197k tokens, and reported the banding without an upload test. So the list itself does a lot. This adds repeatable numbers, the upload test, and fewer tokens. Both missed a face swap they weren't told to look for, so use --one-character.