NeurIPS Experiments

Use this skill before submission or rebuttal when the main question is whether the evidence supports the NeurIPS claim. It is not enough to win a leaderboard; reviewers need to know why the result is scientifically meaningful.

Experiment audit

• Baselines: include strong, current, tuned baselines and explain any missing comparison.
• Ablations: isolate the mechanism, not just remove components at random.
• Robustness: test across seeds, datasets, distribution shifts, scales, hyperparameters, or realistic deployment conditions when relevant.
• Compute: disclose hardware, training time, resource assumptions, and whether comparisons are fair.
• Data: document splits, contamination controls, license, demographic or domain coverage, and privacy/consent limits.
• Negative results: use them to calibrate claims; NeurIPS has a contribution type for negative results, but the bar remains high.
• Use-inspired work: connect results to the real task without turning the paper into an application report with no ML contribution.