Benchmark Runner

Standardizes performance comparison methodology: metric selection, test case design, environment capture, result formatting, and tradeoff analysis. Produces reproducible benchmark reports that support informed decisions — not just "A is faster than B" but "A is faster for small inputs while B scales better."

Reference Files

File	Contents	Load When
`references/metric-selection.md`	Metric catalog (latency percentiles, throughput, memory, accuracy), selection criteria per task type	Always
`references/test-case-design.md`	Representative input selection, scale variation, edge case coverage, warmup strategies	Always
`references/environment-capture.md`	Hardware/software context recording, reproducibility requirements, variance control	Always
`references/statistical-rigor.md`	Sample sizing, variance measurement, significance testing, outlier handling	Results need statistical validation

benchmark-runner

Benchmark Runner

Reference Files

Prerequisites