Where AI Coding Agents Fail: Pre-Flight PR Checker

This skill applies findings from a large-scale empirical study of 33,000 agent-authored pull requests across GitHub (Ehsani et al., MSR 2026) to systematically audit PRs before submission. The study identified that 28.5% of agent PRs fail to merge, with specific, predictable failure patterns. By checking a PR against the paper's hierarchical rejection taxonomy — covering reviewer-level, PR-level, code-level, and agentic-level failures — this skill catches the exact issues that cause real-world rejections before they happen.

When to Use

• When preparing to submit an AI-generated or AI-assisted pull request to an open-source or team repository
• When reviewing a batch of changes and wanting to predict merge likelihood
• When an agent-generated PR has been rejected and you need to diagnose why
• When deciding whether a task (bug fix, feature, refactoring) is suitable for agentic contribution
• When you want to decompose a large PR into smaller, more mergeable units
• When auditing CI/CD readiness of changes before pushing

Key Technique

The paper's core contribution is a hierarchical rejection taxonomy derived from qualitative analysis of 600 rejected PRs, organized into four levels: Reviewer (38% of rejections — PR simply abandoned with no review), PR-level (30% — duplicates, unwanted features, wrong branch), Code-level (22% — CI failures, incorrect/incomplete implementations), and Agentic-level (2% — misalignment with reviewer instructions, license issues). The critical insight is that the majority of agent PR failures are not code quality problems — they are contextual and social failures. The single largest rejection category is "Abandoned/Not Reviewed" (38%), meaning the PR never received meaningful human engagement.

Quantitatively, the study found three statistically significant predictors of rejection: (1) PR size — each increase in lines changed reduces merge odds (Cliff's delta = -0.17); (2) file spread — touching more files correlates with rejection (delta = -0.10); and (3) CI failures — each additional failed CI check reduces merge odds by ~15% (delta = -0.24). Review comment count and revision count were not significant predictors. Task type matters enormously: documentation PRs merge at 84%, CI/build PRs at 74-79%, but bug fixes merge at only 64% and performance PRs at 55%.