Model Compression

This is a Hermes-native model-compression workflow skill.

Why This Exists

model-compression exists because making a model smaller had no owner: omh-inference-serving chooses an engine and the quantized format to serve, and omh-model-finetuning trains for new behavior, while producing the smaller weights and proving they still do the person's task fell between them, and perplexity was the only number anyone checked.

First Steps

• Ask what hardware and memory budget the model must fit and which runtime will load it before naming a method.
• Ask for the held-out task examples and the uncompressed model's score on them.

Do Not Use When

• The ask is choosing a serving engine, which published checkpoint format to serve, or deploying and benchmarking an endpoint; use omh-inference-serving.
• The ask is training a model for new behavior on your own data with SFT, DPO, RLVR or a LoRA adapter; use omh-model-finetuning.
• The ask is turning lessons from OMH sessions into rules or guidance; use omh-rules-distill.
• The ask is onboarding a new model generation into OMH's routing, calibration, or pricing; use omh-model-optimization.