guardrail-metrics-reference

Overview

A guardrail metric is a measure that must not significantly degrade for an experiment to ship, even if the primary metric (the OEC - Overall Evaluation Criterion) improves. The guardrail prevents "we shipped 5% revenue improvement, but latency 30% worse and we discovered too late." Per Kohavi et al. Trustworthy Online Controlled Experiments (Cambridge Univ. Press, ISBN 978-1108724265), this is "the most important class of metrics after the OEC."

This skill is a pure reference consumed by the AB-test validity checklist and the SDK-test skills.

How to use this reference