Tutorial: Debugging CUDA Crashes with Kernel API Logging

This tutorial shows you how to debug CUDA crashes and errors in SGLang using the @debug_kernel_api logging decorator.

Goal

When your code crashes with CUDA errors such as illegal memory access, device-side assert, out-of-bounds, or NaN/Inf, use kernel API logging to:

• Capture input tensors BEFORE the crash occurs
• Understand what data caused the problem
• Track tensor shapes, dtypes, and values through the call boundary that triggered the crash
• Detect numerical issues such as NaN, Inf, or obviously wrong shapes

Why Use Kernel API Logging?

Problem: CUDA errors often crash the program before normal debugging output is flushed.

Solution: SGLang's @debug_kernel_api decorator logs inputs before execution, so you can still see what caused the crash even after the program aborts.

What Is Covered?