Chunk Markdown

Split a Markdown document into semantically meaningful chunks based on heading hierarchy and content classification. Produces numbered files and a manifest index.

When to use

• Markdown file is too long for convenient reading or processing
• Need to organize content by chapter, section, appendix, or index
• Preparing document for downstream processing by section type
• Creating granular pieces for RAG or semantic search indexing

Inputs to gather

• Markdown file path (required)
• Heading depth for split: --depth (optional; defaults to 2, i.e., split at ##)
• Minimum tokens per chunk: --min-tokens (optional; defaults to 200)
• Maximum tokens per chunk: --max-tokens (optional; defaults to 8000)

Procedure