Performance Profiler

Purpose

Identify and fix performance bottlenecks using a measurement-first, data-driven approach. No optimization without data.

Profiling Types

CPU Profiling

Measures time spent in each function. Two modes: sampling (periodically sample the call stack — low overhead, statistical) and instrumentation (instrument every function entry/exit — high overhead, exact). Use sampling for production, instrumentation for dev. Output: flat profile (total time per function) and call graph (caller-callee relationships). Tools: perf, PerfView, dotTrace, Py-Spy, pprof, Chrome DevTools Performance tab.

Sampling profilers work by recording the current call stack at a fixed frequency (e.g., 100 Hz). The number of samples in each function is proportional to the time spent there. Statistical noise decreases with more samples: at 100 Hz for 60 seconds, you get 6000 samples, enough for 1% precision on hot paths. Instrumentation profilers add entry/exit hooks to every function — exact counts but 10-100x overhead.

Memory Profiling

Measures allocation rate, object lifetime, GC pressure, and heap composition. Key metrics: allocation rate (MB/s), GC pause duration, GC frequency, heap size after GC, large object heap size, object retention depth. Memory leaks appear as: heap growing monotonically, Gen 2 objects that should be ephemeral, increasing thread-local storage, unbounded collections in static fields. Tools: dotMemory, Valgrind memcheck, Chrome DevTools Memory tab, Py-Spy, pprof.

Heap analysis workflow: take snapshot, force GC, take post-GC snapshot. Objects remaining after GC that should have been collected are the leak candidates. For each candidate, trace the retention path to find what holds the reference.

I/O Profiling

Measures blocking time on disk, network, and IPC operations. Key metrics: iowait (blocked on storage), read/write latency, IOPS, queue depth, connection pool utilization, socket buffer occupancy. I/O bottlenecks appear as: high iowait % in top, thread pool starvation (all threads blocked on I/O), high variance in response latency under load, connection timeouts and socket exhaustion.