DataChef: Automated Data Recipe Generation for LLM Adaptation

This skill enables Claude to generate end-to-end data recipes — executable Python pipelines that transform raw data sources into optimized training corpora for fine-tuning LLMs on specific target tasks. Based on the DataChef framework, the approach treats data curation as a structured optimization problem: given a target benchmark and a pool of available data sources, produce a complete pipeline of filtering, synthesis, mixing, and augmentation steps that maximizes downstream task performance. Instead of manual trial-and-error data curation, this skill applies principled recipe design with proxy-based evaluation to iterate toward high-quality training data.

When to Use

• When the user wants to fine-tune or adapt an LLM to a specific domain (math, code, finance, medical, etc.) and needs to decide which datasets to use and how to process them
• When the user has multiple raw data sources and needs a pipeline to filter, clean, combine, and format them into a training corpus
• When the user asks "what data should I train on?" or "how do I mix these datasets for best results?"
• When the user needs to synthesize additional training examples to fill gaps in their existing data
• When the user wants to automate the data preparation step of an LLM training workflow
• When the user is iterating on data quality and needs a systematic approach rather than ad-hoc filtering

Key Technique

Data recipes as code. A data recipe is a tuple r = (g, d) where g is an executable Python pipeline and d is the resulting training dataset. The pipeline composes operations from a pool: filtering (quality-based selection, deduplication, keyword extraction), synthesis (LLM-generated augmentation, format conversion), mixing (weighted combination of multiple sources), and selection (relevance-based subsetting). The key insight is that the entire recipe — not just individual steps — must be optimized holistically, because interactions between processing steps matter.

Proxy reward for fast iteration. Evaluating a recipe normally requires full fine-tuning, which is prohibitively expensive. DataChef uses a Data Verifier — a rubric-based evaluator that scores sampled instances from the generated dataset on a scale: Invalid/Incorrect (0), Task Mismatch (0.4), Pass (1.0). This proxy score correlates with downstream performance and enables evaluating dozens of candidate recipes without training a model for each one. Penalties are applied for execution failures (empty dataset) and format violations.