compileiq-debug

A symptom-indexed cheat sheet. Find the row that matches what the user is seeing, follow the first action, then dig into the matching detail section.

Symptom table

Symptom	Most-likely cause	First action
Search hangs on first eval; socket timeout	Search space too large for default `CIQ_SOCKET_TIMEOUT=20`; OR release fetch slow/blocked; OR `forkserver` issue	Raise `CIQ_SOCKET_TIMEOUT=120`; if still hangs, `CIQ_SEARCH_SPACES_DIR=<local mirror>`; if still hangs, `CIQ_PROCESS_MODE=spawn`. Do NOT symlink BLAS — that fix is obsolete
Every eval returns `INVALID_SCORE`	Return-type mismatch (tuple vs scalar) / correctness gate / `task_timeout` too tight	`Search.sample(1)` + call objective by hand; check `num_objectives` vs return shape; raise `task_timeout`
Every eval returns the same score	ACF not reaching the compiler — framework cache is hiding it	Apply Debug-pack `O0` ACF by hand; if the score doesn't regress, fix cache-bust (`TRITON_ALWAYS_COMPILE=1`, `HELION_SKIP_CACHE=1`, fresh `TRITON_CACHE_DIR`, drop FlashInfer cubin packages)
`TypeError: fromhex() … not dict`	Legacy `bytes.fromhex(config_blob)` in objective	Replace with `save_compiler_config(acf_path, config)` — see `compileiq-author-objective`
`"not in expected format"`	Objective returned wrong shape	`num_objectives` must equal `len(return_tuple)`; scalar return only when `num_objectives=1`
Convergence stalled (best score flat)	Pool too small for space; mutate_rate too low; or kernel near-optimal	Raise `pool_size`; raise `mutate_rate`; sample diversity with `Search.sample(20)`
Increasing invalid rate over generations	Mutation arm spreading; compiler version drift mid-run	`CIQ_KEEP_CACHE=1`, re-run, inspect failing configs offline
CV% > 10% on validation	Unlocked clocks / thermal throttling / GPU contention	Lock GPU + memory clocks; pin `CUDA_VISIBLE_DEVICES`; watch `nvidia-smi dmon` for thermals
Need to know why a winning ACF helps	Profile with NCU	See NCU section below