JSON Data File Validation Test Design

Extracted: 2026-02-11 Context: Validating a large JSON data file (exam questions) generated by a build script against its schema, source data, and business rules

Problem

JSON data files generated by scripts (from text, CSV, API, etc.) can contain subtle issues:

• Stray characters from OCR/copy-paste (e.g., ß mixed into Japanese text)
• Schema violations that the app silently swallows
• Cross-reference mismatches (source data vs generated output)
• Missing or duplicate entries
• Business rule violations (e.g., correct answer not in choices)

Manual review of large files (60+ entries, 3000+ lines) is unreliable.

Solution: Layered Pytest Validation

Structure tests in layers from structural to semantic: