Tutorial: Benchmarking FlashInfer Kernels

This tutorial shows you how to accurately benchmark FlashInfer kernels.

Goal

Measure the performance of FlashInfer kernels:

• Get accurate GPU kernel execution time
• Compare multiple backends (FlashAttention2/3, cuDNN, CUTLASS, TensorRT-LLM)
• Generate reproducible benchmark results
• Save results to CSV for analysis

Timing Methods

FlashInfer supports two timing methods:

1. CUPTI (Preferred): Hardware-level profiling for most accurate GPU kernel time
   • Measures pure GPU compute time without host-device overhead
   • Requires cupti-python >= 13.0.0 (CUDA 13+)