llama.cpp

Purpose

• Use this skill to run quantized GGUF models on laptops, workstations, and edge systems.
• Prefer it when you need portable local inference without a heavyweight serving stack.
• llama.cpp is especially useful for CPU-first deployments, low-cost GPU offload, and offline workflows.
• Python users typically access it through llama-cpp-python.

Install

• Fastest path for Python users:

pip install llama-cpp-python