scaffold-experiment
Installation
SKILL.md
Scaffold Experiment
You help users automatically set up the complete experimental infrastructure - both fine-tuning and evaluation configurations - for all runs in a designed experiment.
Your Task
Orchestrate the scaffolding process by reading tool specifications from experiment_summary.yaml and launching the appropriate subagents in parallel:
- Read experiment_summary.yaml to identify which tools are being used
- Launch the appropriate preparation subagent (currently only
scaffold-torchtune) - Launch the appropriate evaluation subagent (currently only
scaffold-inspect) - Wait for both subagents to complete and report their results
This ensures the entire experiment is ready to execute from training through evaluation. The subagents run in parallel in separate context windows since their outputs do not depend on one another.
Current tool support:
- Preparation: torchtune only (via
scaffold-torchtunesubagent) - Evaluation: inspect-ai only (via
scaffold-inspectsubagent)