integrating-models
Installation
SKILL.md
Goal
Integrate a new model into diffusers end-to-end. The overall flow:
- Gather info — ask the user for the reference repo, setup guide, a runnable inference script, and other objectives such as standard vs modular.
- Confirm the plan — once you have everything, tell the user exactly what you'll do: e.g. "I'll integrate model X with pipeline Y into diffusers based on your script. I'll run parity tests (model-level and pipeline-level) using the
parity-testingskill to verify numerical correctness against the reference." - Implement — write the diffusers code (model, pipeline, scheduler if needed), convert weights, register in
__init__.py. - Parity test — use the
parity-testingskill to verify component and e2e parity against the reference implementation. - Deliver a unit test — provide a self-contained test script that runs the diffusers implementation, checks numerical output (np allclose), and saves an image/video for visual verification. This is what the user runs to confirm everything works.
Work one workflow at a time — get it to full parity before moving on.
Setup — gather before starting
Before writing any code, gather info in this order:
- Reference repo — ask for the github link. If they've already set it up locally, ask for the path. Otherwise, ask what setup steps are needed (install deps, download checkpoints, set env vars, etc.) and run through them before proceeding.
- Inference script — ask for a runnable end-to-end script for a basic workflow first (e.g. T2V). Then ask what other workflows they want to support (I2V, V2V, etc.) and agree on the full implementation order together.
- Standard vs modular — standard pipelines, modular, or both?