collect-independent-judgments

Purpose

Collect blinded judgments from genuinely independent evaluators or perspectives before exposing peer outputs.

Input contract

required: [judgment_prompt, evaluator_roster, candidate_set, independence_rules]
optional: [rubric, response_deadline, abstention_policy]
constraints: [evaluators receive no peer judgments; each ballot preserves rationale and provenance]

Procedure

1. Freeze the candidate set, rubric, prompt, and independence rules before collection.
2. Obtain one sealed judgment and rationale per evaluator, recording abstentions and unavailable evidence.
3. Check for shared data, model, prompt, or deliberation leakage before unblinding.
4. Unblind only after collection, then return ballots, independence flags, and an aggregation-ready dataset.

If independent judgments are complete and their provenance is preserved, consider aggregate-ranking as the next tactic.