Model-Evaluation Skill

Purpose

This skill makes a medical-imaging model's held-out evaluation task-correct and honest: the right metric for the task and the prevalence, with uncertainty, calibration, and subgroup performance. It emits a per-case metric table that the publication statistics build on, and gates the metric choice against Metrics Reloaded (Maier-Hein & Reinke et al., Nat Methods 2024) and CLAIM 2024.

It sits between /model-validation (which audits the split / design) and /analyze-stats (which owns the comparative inference). It computes the imaging-specific per-case metrics (surface distances, FROC, ECE of a softmax head); /analyze-stats owns DeLong / NRI / IDI / decision curves / MRMC. Like /analyze-stats, it generates and executes code on your predictions — numbers are never hand-typed.

When to use

• You have held-out predictions + ground truth and need task-correct metrics with CIs, calibration, and subgroup slices, plus a per-case table for the manuscript statistics.