Inference Serving

This is a Hermes-native inference-serving workflow skill.

Why This Exists

inference-serving exists so serving an LLM runs as one decided, gated, measured process instead of scattered flag folklore: the engine choice is a table, the deployment is an idempotent runbook whose only completion evidence is the observed verification, and the benchmark speaks the standard metric vocabulary.

Do Not Use When

• A new model generation needs recognition, calibration, routing, and pricing onboarding; use omh-model-optimization.
• The user wants their own machine's model routing or providers configured; use omh-model-setup.
• The question is whether a coding runtime/executor can run at all; use omh-executor-runtime-readiness.
• The goal is application or system performance rather than the serving endpoint itself; use ulw-perf.
• The ask is producing the smaller weights -- GPTQ or AWQ calibration, pruning, or a distilled student -- and proving they still do the task; use omh-model-compression.

Examples

Good example: