dspy-ragas
Installation
SKILL.md
Ragas — Decomposed RAG Evaluation for DSPy
Guide the user through evaluating DSPy RAG pipelines with Ragas, an evaluation framework that decomposes RAG quality into independent metrics for retriever and generator.
What is Ragas
Ragas is an open-source evaluation framework (12.9k+ GitHub stars, Apache 2.0) purpose-built for RAG pipelines. Instead of a single accuracy score, it breaks evaluation into decomposed metrics:
| Metric | What it measures | Needs ground truth? | Evaluates |
|---|---|---|---|
| Faithfulness | Is the answer grounded in retrieved context? | No | Generator |
| AnswerRelevancy | Does the answer address the question? | No | Generator |
| ContextPrecision | Are relevant docs ranked higher? | Yes (reference) | Retriever |
| ContextRecall | Did retrieval find all relevant info? | Yes (reference) | Retriever |
| AnswerCorrectness | Does the answer match the reference? | Yes (reference) | End-to-end |
This decomposition tells you where your RAG pipeline fails — retriever or generator — so you know what to fix.