RAG & Search Engineering — Complete Reference

Build production-grade retrieval systems with hybrid search, grounded generation, and measurable quality.

This skill covers:

• RAG: Chunking, contextual retrieval, grounding, adaptive/self-correcting systems
• Search: BM25, vector search, hybrid fusion, ranking pipelines
• Evaluation: recall@k, nDCG, MRR, groundedness metrics

Modern Best Practices (Jan 2026):

• Separate retrieval quality from answer quality; evaluate both (RAG: https://arxiv.org/abs/2005.11401).
• Default to hybrid retrieval (sparse + dense) with reranking when precision matters (DPR: https://arxiv.org/abs/2004.04906).
• Use a failure taxonomy to debug systematically (Seven Failure Points in RAG: https://arxiv.org/abs/2401.05856).
• Treat freshness/invalidation as first-class; staleness is a correctness bug, not a UX issue.
• Add grounding gates: answerability checks, citation coverage checks, and refusal-on-missing-context defaults.
• Threat-model RAG: retrieved text is untrusted input (OWASP LLM Top 10: https://owasp.org/www-project-top-10-for-large-language-model-applications/).