Guidance — Constrained LLM Generation

Overview

Guidance (guidance-ai/guidance, version 0.3.x) constrains the token sampling itself, so output always matches a regex, a fixed list of options, a JSON schema or a context-free grammar. Model objects are immutable: lm += ... returns a new model with the text and any named captures. Backends: Transformers, LlamaCpp, OnnxRuntimeGenAI, OpenAI, Azure helpers (create_azure_openai_model, create_azure_aifoundry_model) and Mock for tests. Constraints are fully enforced only where the backend gives token-level control, i.e. local models; hosted APIs restrict what can be expressed.

Instructions

Install

pip install guidance          # backend packages (transformers, torch, llama-cpp-python) are installed separately

Chat models: roles, capture, regex, select

from guidance import system, user, assistant, gen, select
from guidance.models import Transformers