sglang-runtime-context
Installation
SKILL.md
SGLang runtime-context architecture
One container owns process-static runtime state: sglang.srt.runtime_context.RuntimeContext
(a process singleton reached via get_context()). Everything below is a tier on it.
| Tier | Accessor | Holds | Lifecycle |
|---|---|---|---|
| raw config seed | get_server_args() |
the published ServerArgs — the startup record, for debugging, dumps and provenance. Business code does not read fields off it: the read ratchet pins that at zero, and "Reading config: the seed is off limits" below says what to read instead, which forms the ratchet sees, and what is outside it by construction (a runtime-computed name; a whole-object hand-off) |
published at process entry; re-publish is last-publish-wins (the tokenizer publish in the launcher process; sequential engine rebuild in one process, e.g. unit tests) and re-projects the bags; read-only |
| resolved config | get_exec() get_memory() get_schedule() get_model() get_spec() get_serving() get_observability() get_disagg() get_lora() get_mm() get_device() |
namespace config bags — the single source of truth for resolved config; leaves are real attributes (dynamo-traceable). Each is a module function of no arguments, and a module binds the name once: manager.get_disagg(), self.get_disagg = get_disagg, or a same-named import next to the bag one (from model_loader import get_model) all import fine and fail only when that path runs. ruff --select F811 catches the import collision; RuntimeContext has no bag-named member and no __getattr__, so the member-call shapes are an AttributeError at call time — give it a delegating __getattr__ and they go silent instead |
projected at publish from the declarations over server_args' raw fields; mutated only via get_context().override |
| runtime flags | get_flags() |
state that is not a pure function of config: capture (cuda-graph lifecycle), moe (ACTIVE backends, swappable), dp (DP-attention runtime flags) |
materialized at subsystem init; groups offer override() for tests |
| resources | get_resources(), get_stream(name), get_buffer(name, factory) |
process-level handles: graph pools, EPLB state, EP dispatcher state, named side streams, workspace buffers | lazy; cleared by reset_context() |
| per-forward | get_forward() |
forward-scoped flags (multi-stream switch, MoE output buffer, attn-TP inputs, extend-in-batch) | contextvar-backed; scoped(**kw) restores on exit; new threads see defaults |
| parallel | get_parallel() |
one spelling per name: ranks and group handles are the live topology (@property, read-through); every other name, sizes included, is a leaf of the parallel config bag |
ranks/groups: after dist init; leaves: after publish |
reset_context() (unit-test teardown) drops the published config and installs fresh
flags/resources/forward tiers.