HQQ - Half-Quadratic Quantization

Fast, calibration-free weight quantization supporting 8/4/3/2/1-bit precision with multiple optimized backends.

When to use HQQ

Use HQQ when:

• Quantizing models without calibration data (no dataset needed)
• Need fast quantization (minutes vs hours for GPTQ/AWQ)
• Deploying with vLLM or HuggingFace Transformers
• Fine-tuning quantized models with LoRA/PEFT
• Experimenting with extreme quantization (2-bit, 1-bit)

Key advantages:

• No calibration: Quantize any model instantly without sample data
• Multiple backends: PyTorch, ATEN, TorchAO, Marlin, BitBlas for optimized inference
• Flexible precision: 8/4/3/2/1-bit with configurable group sizes
• Framework integration: Native HuggingFace and vLLM support
• PEFT compatible: Fine-tune quantized models with LoRA