PaperSearchQA: Iterative Search-Reason Agents for Scientific Literature

This skill teaches Claude to build search-reason agents — systems that iteratively formulate queries, retrieve scientific documents, reason over results, and self-verify answers before responding. Based on the PaperSearchQA method (Burgess et al., EACL 2026), the core insight is that agents trained with reinforcement learning on verifiable rewards (RLVR) learn emergent behaviors — planning search queries, leveraging parametric knowledge before retrieval, and self-verifying answers — that outperform static RAG pipelines by 10-15 accuracy points on technical QA. This skill applies that architecture pattern to build practical search agents over any document corpus.

When to Use

• When the user wants to build a QA system that searches over a corpus of scientific papers or technical documents
• When the user needs an agent that can perform multi-step retrieval — where the first search informs subsequent queries
• When building a RAG pipeline and the user wants to upgrade from single-shot retrieval to iterative search-reason loops
• When implementing a literature review agent that must find, cross-reference, and synthesize information from multiple papers
• When the user wants to construct a factoid QA dataset from a document corpus for training or evaluation
• When building an "AI Scientist" assistant that answers technical questions with cited evidence from a paper collection

Key Technique

The Search-Reason Loop. Unlike standard RAG (retrieve once, generate once), PaperSearchQA agents operate in an interleaved loop: <think> to reason, <query> to search, receive <information>, then repeat as needed before producing an <answer>. The agent decides when to search, what to search for, and when it has enough evidence to answer. This is implemented by intercepting generation at <query> tokens, executing retrieval, injecting results as <information> blocks, and resuming generation. The agent learns to decompose complex questions into targeted sub-queries.

RLVR Training with GRPO. The agent is trained using Group Relative Policy Optimization (GRPO), where the only reward signal is whether the final answer matches the ground truth (exact match after normalization). No intermediate supervision is needed — the agent discovers effective search strategies through exploration. This is key: you don't need to label which documents are relevant or which queries are good. You only need question-answer pairs. The training uses a KL divergence penalty against a reference model to prevent reward hacking.