ai-training-data-generation
AI Training Data Generation
Overview
A comprehensive skill for automatically generating high-quality training datasets from documents, text corpora, and structured content. Optimized for low-resource languages, dictionary content, and domain-specific knowledge extraction.
Key Sources: Dictionary databases, Bible EPUBs (NWT), JW brochures, parallel text corpora
Capabilities
- Multi-strategy Generation: Dictionary pairs, contextual definitions, completion tasks, classification examples
- Quality Filtering: Confidence scoring, duplicate removal, and content validation
- Format Flexibility: Support for multiple AI training formats (JSONL, HuggingFace, Ollama, OpenAI)
- Language Awareness: Multi-language support with special handling for accented characters
- Scalable Processing: Generate thousands of examples from large documents
- Balance Management: Ensure dataset diversity and prevent category imbalance
- EPUB Processing: Extract parallel verses from Bible EPUBs for translation training
- Sentence Alignment: Align parallel sentences from bilingual documents
More from findinfinitelabs/chuuk
large-document-processing
Process large documents (200+ pages) with structure preservation, intelligent parsing, and memory-efficient handling. Also covers intelligent text chunking for AI training and RAG systems. Use when working with complex formatted documents, multi-level hierarchies, or when splitting large content for AI pipelines.
28python-venv-management
Automatically manage Python virtual environments (.venv) in terminal commands. Always activate .venv before running Python/pip commands. Supports macOS, Linux, and Windows with shell-aware activation. Use when executing Python scripts, installing packages, or running development servers. Critical for consistent environment management.
14bible-epub-processing
Parse and extract structured content from Bible EPUBs (NWT) for parallel text alignment between Chuukese and English. Use when working with Bible data, verse extraction, parallel corpus building, or generating training data from Scripture.
14security-environment-standards
Security and environment configuration standards for web applications, including environment variable management, secure coding practices, and production deployment security. Use when setting up environments, configuring security, or deploying applications.
13intelligent-text-chunking
Split large texts into meaningful, AI-optimized chunks while preserving semantic coherence and document structure. Covered by the large-document-processing skill — see that skill for full details.
13document-ocr-processing
Process scanned documents and images containing Chuukese text using OCR with specialized post-processing for accent characters and traditional formatting. Use when working with scanned books, documents, or images that contain Chuukese text that needs to be digitized.
12