Benchee Profiling

RULES — Follow these with no exceptions

1. Benchmark in production-like conditions — use MIX_ENV=prod for realistic results
2. Compare alternatives — always benchmark at least 2 approaches to justify optimization; ensure implementations do the same thing
3. Document performance regressions — track benchmark results over time in bench/baseline.json with a >10% threshold
4. Profile before benchmarking — use :fprof or :eprof to identify the actual bottleneck before writing benchmarks
5. Use multiple inputs — test with small, medium, and large inputs to catch size-dependent behavior; use realistic data sizes
6. Warm up before measuring — run warmup phase to let JIT compilation settle
7. Run sufficiently and repeatedly — use time: 10 (10 seconds) for reliable measurements and run 3-5 times to rule out variance
8. Separate I/O benchmarks — never benchmark network or disk I/O in the same run as compute benchmarks

Workflow