Detecting and Correcting Hallucinations in LLM-Generated Code via AST Analysis

This skill enables Claude to apply a deterministic, static-analysis post-processing pipeline to LLM-generated Python code. Rather than relying on execution or probabilistic LLM self-repair, the technique parses code into an Abstract Syntax Tree (AST), builds a Knowledge Base (KB) of legitimate API signatures via Python introspection (inspect, dir(), getattr()), and uses deterministic rules to detect and auto-correct Knowledge Conflicting Hallucinations (KCHs) — subtle semantic errors like non-existent parameters, fabricated method names, and wrong argument counts that pass linters but cause runtime failures.

When to Use

• When the user asks you to validate generated Python code against actual library APIs before running it
• When code uses third-party libraries (pandas, numpy, requests, sklearn, etc.) and the user suspects hallucinated parameters or methods
• When debugging a TypeError: unexpected keyword argument or AttributeError: has no attribute that came from generated code
• When the user wants a pre-execution safety check on code snippets produced by an LLM
• When reviewing code that makes many API calls and the user asks "are all these parameters real?"
• When auto-correcting code that has plausible-looking but non-existent API arguments (e.g., pd.read_csv(file, delimeter=',') — misspelled delimiter)