Extraction Pipeline Patterns

Xberg's format detection -> extraction -> fallback orchestration for 75+ file formats

Core Pipeline Architecture

The extraction pipeline (crates/xberg/src/core/pipeline.rs, crates/xberg/src/extraction/) orchestrates:

1. Format Detection - MIME type inference + extension validation -> select appropriate extractor
2. Intelligent Extraction - Route to format-specific extractors (PDF, DOCX, Excel, HTML, images, archives, etc.)
3. Fallback Strategies - Password-protected PDFs, OCR for images, nested archive handling, corrupted file recovery
4. Post-Processing Pipeline - Validators, quality processing, chunking, custom hooks (see core/pipeline.rs)

Format Detection Strategy

Location: crates/xberg/src/core/mime.rs, crates/xberg/src/core/formats.rs

Pattern: detect via magic bytes, validate extension alignment (prevent spoofing), route to extractor. Multiple extractors for same format -> choose highest confidence/specificity.