usenixsec-experiments
Installation
SKILL.md
USENIX Security Experiments
The evaluation is where USENIX Security papers are won or lost, and the committee
reads it as an adversary would: looking for the experiment you did not run because
it would have hurt. This skill audits security evaluations against the venue's
specific rigor bars. It pairs with usenixsec-reproducibility (making runs
regenerable) and usenixsec-writing-style (reporting them).
Match the experiment to the claim type
| Claim type | The experiment reviewers demand | The usual gap |
|---|---|---|
| Attack | End-to-end demonstration on a realistic target, success rate over trials | Works only in a toy setup; success rate is one lucky run |
| Defense | Adaptive attacker who knows the defense, plus overhead | Evaluated only against the original, non-adaptive attack |
| Detection | Detection rate and false-positive rate on realistic base rates | FPR measured on a clean dataset, not deployment traffic |
| Measurement | Cross-vantage / cross-time validity of the finding | Single vantage, single snapshot, over-generalized |
| System/protocol | Correctness + performance vs a credible baseline | Baseline is a strawman or an unoptimized reimplementation |