llama.cpp Operations

Treat every launch recipe as a hypothesis about a specific build, model, host, and workload. Discover capabilities from the installed binary, inspect the model and startup logs, then measure the requested boundary.

Operating contract

1. Record the exact llama.cpp version or commit, installation method, OS and architecture, CPU and RAM, accelerator and memory, driver/toolkit, available devices, model provenance and quantization, intended context, concurrency, and workload.
2. Read the installed command's --help before using a flag from documentation. llama.cpp flags, defaults, binary names, and REST behavior change frequently.
3. Confirm the target, scope, and rollback path before acting. Read-only discovery may proceed without confirmation.
4. Verify the backend from --list-devices and model-load logs. A successful build or an accepted GPU flag does not prove acceleration is active.
5. Start with a bounded CLI smoke test on loopback or local input. Establish a measured baseline before changing threads, batches, context, cache types, offload, or split mode.
6. Call work complete only at the requested boundary: binary, model load, generated output, API response, benchmark comparison, or diagnosed failure with evidence.

When not to use

Use ml-engineering for model training, fine-tuning, broad quantization methodology, evaluation design, or choosing among llama.cpp, vLLM, TGI, and other engines. Use the relevant product skill for Ollama, LM Studio, or LlamaIndex. Use binding-specific documentation for llama-cpp-python, node-llama-cpp, or other language wrappers.

Read-only preflight