compute-mamba-ratio

Installation
SKILL.md

Optimal hybrid dual-pool ratio (--mamba-full-memory-ratio)

A hybrid model (attention layers + linear-attention layers — the recurrent-state family: Mamba/SSM, GDN, KDA, etc.) splits serving memory into two independently-budgeted pools, fixed once at startup:

  • state pool (the linear-attention recurrent state) → caps concurrency (hard: whole slots, worst-case reserved, fail-loud)
  • full-KV pool (attention KV) → caps context × concurrency (soft: paged, over-committable via retraction)

--mamba-full-memory-ratio r splits the post-weight budget: mamba_budget = rest · r/(1+r), i.e. mamba_budget : kv_budget = r. This skill picks the r (or the pin---max-mamba-cache-size alternative) at which neither pool bottlenecks first for the user's workload.

The formula

r*  =  (S + D) · token_equiv · dcp_size / L
token_equiv  =  state_bytes_per_slot / kv_bytes_per_token
Installs
3
GitHub Stars
35.7K
First Seen
Aug 3, 2026
compute-mamba-ratio — sgl-project/sglang