Colibri

Assist with Colibri — the pure-C inference engine for running GLM-5.2 (744B Mixture-of-Experts) on consumer machines with ~25 GB RAM, featuring expert disk-streaming, MLA attention, and native speculative decoding (MTP).

When to use this skill

• Setup & build: Installing C build tools, running setup.sh, configuring OpenMP, cross-platform compilation (Linux, macOS, Windows via WSL)
• Model download & conversion: Running coli convert to fetch GLM-5.2-FP8 shards, convert to int4 container, prepare MTP heads (int8 quantization)
• Inference modes: Running chat mode (coli chat), API server setup, batch inference, configuring sampling and temperature
• Expert caching & storage: Tuning LRU cache sizes, pin-hot-store configuration, disk I/O optimization, thermal management
• Speculative decoding (MTP): Enabling/disabling multi-token prediction, measuring draft acceptance rates, optimizing verification latency
• GPU acceleration: Configuring CUDA pinned experts, GPU vRAM budgeting, hybrid CPU-GPU inference
• Performance profiling: Measuring tokens/second, analyzing expert-load patterns, memory residency tuning
• Integration & deployment: Embedding Colibri in applications, wrapping inference API, local-first LLM pipelines

When not to use this skill