data-pipeline
Installation
SKILL.md
Data Pipeline
4-stage V2 charity evaluation pipeline with 100-point scoring.
Philosophy: Capture broadly, filter later. Correctness > cost, but we can have both.
Quick Reference (V2 Pipeline)
| Stage | Entry Point | What It Does |
|---|---|---|
| 1. Crawl | crawl.py |
Collect data from 5 sources |
| 2. Process Data | process_data.py |
Derive fields + reconcile sources |
| 3. Process Baseline | process_baseline.py |
Generate baseline narratives + export + verify |
| 4. Process Rich | process_rich.py |
Generate rich narratives + export + verify |
Wrapper: ./run_v2.sh runs all 4 stages