Generate a RAG Evaluation Dataset

This recipe analyzes your RAG knowledge base and generates a comprehensive Q&A evaluation dataset.

Step 1: Analyze the Knowledge Base

Read the codebase to find the knowledge base:

• Document files (PDFs, markdown, text files)
• Database schemas (if documents are stored in a DB)
• Vector store configuration (what's being embedded)
• Chunking strategy (how documents are split)

Read every document you can access. Understand:

• What topics does the knowledge base cover?
• What's the depth of information?
• What terminology is used?
• What are the boundaries (what's NOT covered)?