dependency-verification
Installation
SKILL.md
A package name the model produced is a claim, not a fact. The registry is the fact. Verify before you install.
Dependency Verification
Overview
Code-generating language models recommend packages that do not
exist at a measured rate of 5.2% (commercial models) to 21.7%
(open models) across 576,000 samples (Spracklen et al. 2024,
arXiv 2406.10279). Worse, 58% of hallucinated names recur across
reruns, so an attacker can predict them, register the empty name,
and ship malware. This is "slopsquatting." A proof-of-concept
package (huggingface-cli) drew over 30,000 downloads after being
registered against a commonly hallucinated name. Package
hallucination is also inversely correlated with coding-benchmark
score, so a better model does not make this go away.