Usage

/model-migrate-flagos <model_name> [upstream_folder] [plugin_folder]

Argument	Required	Default
`model_name`	Yes	—
`upstream_folder`	No	`/tmp/vllm-upstream-ref`
`plugin_folder`	No	current working directory

Execution

Step 1: Parse arguments and validate paths

Extract from user input:

{{model_name}} = first argument (required, snake_case)
{{upstream_folder}} = second argument or /tmp/vllm-upstream-ref
{{plugin_folder}} = third argument or current working directory

If {{upstream_folder}} doesn't exist, ask user whether to clone it. If {{plugin_folder}} doesn't exist, error out.

→ Tell user: Confirm parsed model name and paths.

Step 2: Load references and resolve placeholders

Read these files (relative to this SKILL.md):

references/procedure.md — step-by-step migration procedure
references/compatibility-patches.md — 0.13.0 patch catalog
references/operational-rules.md — communication, TaskList, bash rules, resilience

The procedure references executable scripts in scripts/:

scripts/validate_migration.py — automated code review (Step 6)
scripts/benchmark.sh — benchmark verification (Step 9)
scripts/serve.sh — serve model locally (Step 10.1, also used for E2E)
scripts/request.sh — test request (Step 10.2)
scripts/e2e_eval.py — E2E correctness verification (Step 11)
scripts/e2e_test_prompts.json — test prompts for E2E (5 text + 5 multimodal)
scripts/e2e_config.template.json — E2E config template (copy to e2e_config.json and fill in)
scripts/e2e_remote_serve.sh — manage GT server on remote machine via SSH

Then investigate upstream source + HuggingFace to resolve all placeholders:

Placeholder	How to derive
`{{model_name}}`	Direct from argument
`{{model_name_lower}}`	Lowercase of model_name (usually identical, e.g. `qwen3_5`) — used in file paths
`{{MODEL_DISPLAY_NAME}}`	From upstream code or HF model card
`{{ModelClassName}}`	From upstream model class (PascalCase)
`{{model_type}}`	From HF config.json `model_type` field
`{{ConfigClassName}}`	From upstream or derive from model_type
`{{skill_root}}`	Absolute path to this skill's folder (the directory containing this SKILL.md)

Naming conventions vary per model — always verify from actual source, never guess.

→ Tell user: Present all resolved values. Use AskUserQuestion if anything is ambiguous.

Step 3: Execute procedure

With placeholders resolved, execute every step in procedure.md sequentially. Apply patches from compatibility-patches.md during the copy-then-patch step. Follow operational-rules.md throughout.

→ Tell user: Before starting, output a numbered plan. Report progress at each step boundary.

Scripts Reference

Script	Step	Description
`validate_migration.py`	6	Automated import/API/registration checks
`benchmark.sh`	9	`vllm bench throughput` with dummy weights
`serve.sh`	10, 11	Start local vLLM server (port 8122, `VLLM_FL_PREFER_ENABLED=false`)
`request.sh`	10	Quick smoke-test request
`e2e_eval.py`	11	Token-level comparison vs upstream GT server
`e2e_test_prompts.json`	11	5 text + 5 multimodal test prompts
`e2e_config.template.json`	11	Config template (GT machine, local port, eval params)
`e2e_remote_serve.sh`	11	SSH-based GT server lifecycle (start/stop/status/logs)

Examples

Example 1: Typical new model

User says: "/model-migrate-flagos kimi_k25"
Actions:
  1. Parse → model_name=kimi_k25, defaults for upstream/plugin paths
  2. Clone upstream, find vllm/model_executor/models/kimi_k25.py
  3. Discover it wraps DeepseekV2 → follow kimi_k25 (wrapper) pattern
  4. Copy file, apply P1+P2 patches, create config bridge
  5. Register, validate, test, benchmark, serve+request
  6. E2E verification against upstream GT
Result: kimi_k25 fully working in plugin, all 11 steps passed

Example 2: Re-run after upstream update

User says: "migrate qwen3_5 again, upstream updated"
Actions:
  1. Idempotent re-run — overwrite existing files with fresh upstream copy
  2. Re-apply patches, re-validate, re-test
  3. Re-run E2E to confirm no regression
Result: qwen3_5 updated to match latest upstream, no regressions

Troubleshooting

General principle: When any runtime error occurs, first compare vLLM upstream code against both the plugin adaptation and the installed 0.13.0 environment. The diff is the fastest path to root cause. See operational-rules.md § Debugging Priority: Upstream-First for the full protocol.

Problem	Typical Cause	Fix
`ImportError` after copy-then-patch	Missing P1 fix (relative→absolute imports)	Verify all `from .xxx` converted to `from vllm.` or `from vllm_fl.`
`AttributeError: module 'vllm' has no attribute X`	API doesn't exist in 0.13.0	Check P3 in compatibility-patches.md; stub or remove
Config not recognized by vLLM	model_type mismatch or config bridge missing	Verify `_CONFIG_REGISTRY[model_type]` matches HF config.json exactly
Registration has no effect	Class name or import path typo	Compare with existing registrations in `__init__.py`
Benchmark `KeyError` on config field	Config bridge missing a field	Compare upstream config class vs bridge; add missing fields with defaults
Benchmark/Serve fails with OOM or "insufficient memory"	GPUs occupied by other processes	Kill GPU processes: `nvidia-smi --query-compute-apps=pid --format=csv,noheader \| xargs -r kill -9` then retry. Never skip these steps.
Model outputs garbled/gibberish text	`ColumnParallelLinear` used for merged projections with different sub-dimensions (TP sharding mismatch)	Override `__init__` to use `MergedColumnParallelLinear(output_sizes=[...])`. See P8 in compatibility-patches.md
`AssertionError: Duplicate op name`	Child class imports custom op from different module path than parent	Use same import path as parent module (e.g. `vllm_fl.ops.fla` not `vllm_fl.models.fla_ops`). See P11
`AttributeError` on `fused_recurrent_*` during CUDA graph warmup	`__init__` override with `nn.Module.__init__(self)` missed attributes used by inherited `_forward_core`	Create ALL attributes from parent's `__init__`, especially custom ops. See P12
E2E: local server not reachable	`serve.sh` port doesn't match `e2e_config.json` local port	Ensure both use same port (default 8122)
E2E: GT server not reachable	GT machine down or docker/conda env wrong	Check `e2e_remote_serve.sh status` or SSH manually
E2E: early token divergence (first 5 tokens)	Weight loading bug, TP sharding error	Check `load_weights`, `stacked_params_mapping`, MergedColumnParallelLinear
E2E: late minor divergence (token #15+)	Numerical noise from different op implementations	Usually acceptable; document in report
`resolve_op` fails with `VLLM_FL_PREFER_ENABLED=false`	Op not registered in dispatch, no fallback	Add try/except fallback to `flag_gems` in op import code

model-migrate-flagos

FL Plugin — Model Migration Skill