const openai = ai({ name: 'openai', apiKey: 'sk-...' }); const claude = ai({ name: 'anthropic', apiKey: 'sk-ant-...' }); const gemini = ai({ name: 'google-gemini', apiKey: 'AIza...' }); const azure = ai({ name: 'azure-openai', apiKey: 'your-key', resourceName: 'your-resource', deploymentName: 'gpt-5-4-mini' }); const deepseek = ai({ name: 'deepseek', apiKey: 'sk-...' }); const mistral = ai({ name: 'mistral', apiKey: 'your-key' }); const cohere = ai({ name: 'cohere', apiKey: 'your-key' }); const custom = ai({ name: 'openai-compatible', apiKey: process.env.PROVIDER_API_KEY, apiURL: 'https://example.com/v1', config: { model: 'provider/model-name' }, }); const reka = ai({ name: 'reka', apiKey: 'your-key' }); const grok = ai({ name: 'grok', apiKey: 'your-key' });


`name` selects a deployment profile; `config.model` selects a model inside that
deployment. Never infer provider behavior from a model ID. For example,
`name: 'together'` with a `deepseek-ai/...` model uses Together's profile rules,
not DeepSeek's native request shape.

Use `axAIProfiles()` to discover the complete named catalog and
`axGetAIProfile(name)` to inspect endpoint requirements, authentication,
operations, capabilities, model rules, sources, and review dates. Use
`name: 'openai-compatible'` plus `apiURL` for an unlisted custom endpoint;
unknown names are errors.

Profile-only branded classes were removed in the major-version migration. Use
`ai({ name: ... })` for Azure OpenAI, Cohere, DeepSeek, DeepSeek Responses,
Mistral, Reka, and Grok. Retained low-level classes represent genuine
transports/runtimes only; legacy model enum and catalog exports remain usable.

## Typesafe / Jev (TypeScript)

Use the [ax-typesafe skill](https://github.com/ax-llm/ax/blob/main/src/ax/skills/ax-typesafe.md)
for Jev question design, rich criteria, native scoring, transport settings, and
hybrid examples. The two interfaces serve different output contracts:

| Need | Interface |
|---|---|
| Required boolean/class outputs with normal signature-shaped results | `ai({ name: 'typesafe', apiKey, trueThreshold: 0.9 })` and `ax(...).forward()` |
| Native probabilities, structured state/criteria, or Score | `typesafe({ apiKey }).systemOne({ state, questions })` |

Both default to `jev-latest`. Set the adapter's model with `config.model`; set
native defaults with `model`. Native `listModels()` retrieves the provider's
catalog without changing configured Ax model aliases.

Boolean conversion uses `noul >= trueThreshold`; the default is `0.5`, and the
finite threshold must be in `[0, 1]`. It applies to all booleans on that provider
instance, is never sent to the server, and does not affect native probabilities.
Choice preserves the selected label without an automatic confidence cutoff.

Value descriptions from `boolean(true "...", false "...")` and
`class "support, billing"(support "...", billing "...")` become native criteria.
The same annotations render as readable descriptions for conventional providers.
The [signature skill](https://github.com/ax-llm/ax/blob/main/src/ax/skills/ax-signature.md)
covers the string and fluent `.describeValues(...)` forms.

Numeric outputs, including bounded numbers, are unsupported; scoring uses
explicit native rubrics. Freeform text, optional/array/nested outputs, media,
tools, and sampling controls are also unsupported. Ax sends its normal prompt
as state. Use a separate generative program for prose or tools.

Typesafe-only balancers retain schema-required generation. Mixed pools select
Typesafe only when the actual request already contains a supported schema.
Incompatible requests are excluded from fallbacks even with degradation enabled.
Usage and raw adapter answers remain in the existing usage and chat-log APIs.
Python, Java, C++, Go, and Rust also implement these interfaces. Each generated
package includes a Typesafe/Jev skill with its native API syntax.

## Renewable Credentials

Use `credentialProvider` for expiring bearer tokens. The callback runs for each
request attempt and receives `{ profile, operation, method, url }`. Fresh
headers override static authentication headers.

```typescript
const vertex = ai({
  name: 'vertex-ai',
  apiURL: process.env.VERTEX_AI_API_URL!,
  config: { model: 'google/gemma-4-26b-a4b-it-maas' },
  credentialProvider: async ({ operation, url }) => ({
    Authorization: `Bearer ${await tokenSource.fresh({ operation, url })}`,
  }),
});

Authentication-required profiles accept apiKey or credentialProvider.
The hook covers chat, stream, embeddings, Responses, transcription, speech, and retry attempts. A hook error stops before transport.
Do not expect an automatic replay after a completed 401 or 403; generation requests are not safely idempotent.
Keep ADC or cloud-SDK dependencies in the host. Ax core only owns the neutral callback contract.

Vertex capability rules are model-aware. Documented Gemini MaaS IDs prefer native schema. The exact google/gemma-4-26b-a4b-it-maas rule prefers json_object, excludes native schema, defaults thinking to max, writes chat_template_kwargs.enable_thinking, and extracts/replays reasoning_content. Unknown Vertex models remain conservative.

WebLLM is browser-only and requires a host-created WebLLM engine. The host loads or reloads models with WebLLM APIs such as CreateMLCEngine(...); Ax only forwards chat requests to that loaded engine. Do not present WebLLM as a portable AxIR provider or a server-side default.

import { ai, AxAIWebLLMModel } from '@ax-llm/ax';

const engine = await CreateMLCEngine(AxAIWebLLMModel.Llama32_3B_Instruct);
const llm = ai({
  name: 'webllm',
  engine,
  config: {
    model: AxAIWebLLMModel.Llama32_3B_Instruct,
    stream: false,
    supportsFunctions: false,
  },
});

Model Presets

import { ai, AxAIGoogleGeminiModel } from '@ax-llm/ax';

const gemini = ai({
  name: 'google-gemini',
  apiKey: process.env.GOOGLE_APIKEY!,
  config: { model: 'simple' },
  models: [
    { key: 'tiny', model: AxAIGoogleGeminiModel.Gemini35FlashLite, description: 'Fast + cheap', config: { maxTokens: 1024 } },
    { key: 'simple', model: AxAIGoogleGeminiModel.Gemini38Flash, description: 'Balanced' },
  ],
});

await gemini.chat({ model: 'tiny', chatPrompt: [{ role: 'user', content: 'Hi' }] });

Model Catalog

import { axGetSupportedAIModels } from '@ax-llm/ax';

const providers = axGetSupportedAIModels();
const openai = providers.find((provider) => provider.name === 'openai');
console.log(openai?.models[0]?.promptTokenCostPer1M);
console.log(openai?.capabilities.serviceTiers);

const reasoningModel = openai?.models.find(
  (model) => model.capabilities.thinkingLevels.includes('high')
);
console.log(reasoningModel?.capabilities.thinkingLevels);

const textProviders = axGetSupportedAIModels({ type: 'text' });
const embeddingProviders = axGetSupportedAIModels({ type: 'embeddings' });

Use axGetSupportedAIModels() to build provider/model selectors before creating an ai(...) instance. It returns bundled static metadata: provider names, display names, default models, raw AxModelInfo pricing/details, model type ('text', 'embeddings', 'code', 'audio', or 'image'), operation availability, provider-training data use, and normalized capabilities for thinking, thoughts, structured outputs, audio, image output, temperature, top-p, portable thinking levels, and verified service tiers. Provider capabilities describe the default deployment profile; each static model also carries its resolved capabilities.thinkingLevels and capabilities.serviceTiers. Portable thinking levels can collapse onto the same provider-native value. serviceTiers lists verified explicit tiers; serviceTier: 'auto' remains available as the provider-delegated policy.

Provider groups and models are sorted cheapest to most expensive based on bundled input + output token pricing; unpriced models sort last. Dynamic profiles remain useful even when models is empty because their provider-level capability metadata is still returned.

Filter with { type: 'all' | 'text' | 'embeddings' | 'code' | 'audio' | 'image' } or an array of those values. The 'text' filter includes code-capable models; use 'code' to show only code-first models.

Dynamic providers such as Azure OpenAI deployments are marked with isDynamic: true and may have an empty or static-limited model list.

The same AxIR-backed catalog is public in every generated package: Python get_supported_ai_models(), Java Ax.getSupportedAIModels(), C++ get_supported_ai_models(), Go GetSupportedAIModels(...), and Rust get_supported_ai_models() (with get_supported_ai_models_with_options(...) for filters). It includes the named OpenAI-compatible profiles accepted by ai(...), including azure-openai, openrouter, together, fireworks, and the explicit openai-compatible fallback. These entries are usually dynamic, so inspect provider-level capabilities even when models is empty.

Portable Inference Service Tiers

Use the shared serviceTier option with auto, standard, flex, or priority. Set it per call, as an instance option, or on a model-key preset; the narrower setting wins. auto delegates to the provider and is omitted when that provider has no explicit auto value. Ax rejects unsupported explicit tiers before transport.

import { ai, AxAIGoogleGeminiModel } from '@ax-llm/ax';

const gemini = ai({
  name: 'google-gemini',
  apiKey: process.env.GOOGLE_APIKEY!,
  config: { model: AxAIGoogleGeminiModel.Gemini35Flash },
});

const response = await gemini.chat(
  { chatPrompt: [{ role: 'user', content: 'Run this when capacity is free.' }] },
  { serviceTier: 'flex' },
);

Inspect ai.getFeatures(model).serviceTiers for the verified tiers on a named profile or exact model. Named OpenAI, Gemini, Azure, Bedrock, Mistral, Groq, OpenRouter, DeepInfra, Cerebras, Grok, Databricks, and Fireworks profiles map the portable value to their provider dialect. Exact modelInfo.supported metadata can opt a conservative custom deployment into a tier.

The applied provider value is normalized into response.modelUsage.tokens.serviceTier; aliases such as default, on_demand, and performance become standard or priority. Add serviceTierPricing.flex or .priority to AxModelInfo when tier pricing differs, and cost estimates will use the tier that actually served the request. Gemini service tiers remain GenerateContent-only: Vertex AI and Gemini Live reject explicit tiers. Anthropic speed: 'fast' remains a separate API.

Routing And Balancing

Choose the primitive by responsibility:

AxMultiServiceRouter combines model lists and dispatches the model key the caller already selected. It does not select a model.
AxProviderRouter selects a provider by request capability and may degrade unsupported media through configured processors. When the selected provider supports images natively, each image remains an image object with its payload, MIME type, detail level, cache and optimization hints, alt text, and ordering with surrounding text intact.
AxBalancer without a strategy orders equivalent services once with a comparator, retries transient provider failures, and fails over in that order.
AxBalancer with strategy.type: 'adaptive' selects among services exposing the same logical model aliases using learned provider reliability, successful latency, and estimated cost.

Adaptive balancing is operational routing, not semantic prompt-to-model routing. Every provider model behind an alias must be an acceptable substitute for that application. Keep quality evaluation and content-aware model selection outside the balancer.

import { AxBalancer, AxInMemoryBalancerStatsStore } from '@ax-llm/ax';

const statsStore = new AxInMemoryBalancerStatsStore();
const routeKeys = new Map<string, string>([
  [openai.getId(), 'openai-primary'],
  [anthropic.getId(), 'anthropic-primary'],
]);

const llm = AxBalancer.create([openai, anthropic] as const, {
  strategy: {
    type: 'adaptive',
    deadlineMs: 6_000,
    badOutcomeCost: 0.02,
    expectedTokens: { promptTokens: 1_200, completionTokens: 300 },
    namespace: 'support-v1',
    routeKey: (service) => {
      const key = routeKeys.get(service.getId());
      if (!key) throw new Error('Missing stable route key.');
      return key;
    },
    slice: ({ options }) =>
      options?.customLabels?.workflow ?? 'default-workflow',
    statsStore,
    onRoutingEvent: (event) => telemetry.emit('llm.route', event),
  },
});

The score is estimated request cost plus badOutcomeCost times the probability of provider failure or missing deadlineMs. badOutcomeCost and estimated cost must use the same currency or unit. By default, cost uses expectedTokens, the route's concrete model mapping, and getEstimatedCost(); missing catalog pricing contributes zero, while estimateCost can supply application pricing. Failures use an EWMA; successful latency is modeled in log space with a Normal-Inverse-Gamma posterior, and Thompson sampling supplies the deadline risk. Capability filtering still runs before ranking.

Rules:

Reuse the in-memory store across balancers in one process. For multiple processes, implement AxBalancerStatsStore with Redis or an application database; its observe() operation must be atomic.
A custom store requires stable, unique routeKey values. Stats are partitioned by namespace, slice, logical model, and route.
statsStore is decision state. onRoutingEvent is best-effort telemetry and must not be used as the authoritative routing state.
Routing events contain scores, route metadata, and sanitized failure categories, never prompts, responses, or raw provider errors.
Adaptive mode attempts each candidate once. Provider-client retries remain controlled by AxAIServiceOptions.retry.
Streaming can fail over before the first emitted chunk. A mid-stream transient failure is learned but cannot be replayed after partial output; caller cancellation is not recorded.
Adaptive selection applies to chat. Embedding, transcription, and speech keep existing balancer behavior.

See the adaptive balancer example for complete provider setup.

Generated Runtime Cancellation

Generated Python, Java, C++, and Rust packages expose a reusable AxCancellationToken; Go keeps its idiomatic context.Context API. Pass the token/context to chat, streaming, embeddings, transcription, speech, or AxGen/AxAgent/AxFlow forwarding. The same cancellation propagates through provider routers, multi-service routing, balancers, retries, and cancellation-aware custom transports.

Cancellation is thread-safe, one-shot, and first-reason-wins. It stops before a new transport attempt, wakes retry backoff, and terminates an active stream at the next blocking-I/O boundary. The surfaced error is always the non-retryable AxAIServiceAbortedError, so routers and balancers must not retry or fail over. Existing methods remain available; use the added cancellation variants/options when cancellation is needed. Realtime WebSocket turns retain their separate lifecycle.

Chat

const res = await llm.chat({
  chatPrompt: [
    { role: 'system', content: 'You are concise.' },
    { role: 'user', content: 'Write a haiku about the ocean.' },
  ],
});
console.log(res.results[0]?.content);

Meta Muse

Use meta for the Responses API (the recommended default), meta-chat for Chat Completions, or meta-messages for Anthropic-compatible Messages. All three use MODEL_API_KEY; muse-spark-1.3 is the default model.

import { ai, AxAIMetaModel } from '@ax-llm/ax';

const muse = ai({
  name: 'meta',
  apiKey: process.env.MODEL_API_KEY!,
  config: { model: AxAIMetaModel.MuseSpark13 },
});

const result = await muse.chat({
  chatPrompt: [{ role: 'user', content: 'Summarize the attached brief.' }],
});

thinkingTokenBudget: 'highest' maps to Meta xhigh; none is rejected. Responses reasoning IDs, summaries, encrypted content, and message phases are retained in Ax chat memory so stateless multi-turn replay works. Streaming Ax programs also retain the final-answer phase. Responses failure events raise errors; token-limited streams are not accepted as completed program output. Generated Responses clients retain parallel tool calls and their provider call IDs, and completion events do not repeat text or arguments already streamed. Assistant turns containing only generated images or encrypted reasoning can be replayed without adding placeholder text.

All three Spark profiles support function-based structured output. When Ax selects its internal __axOutput tool, the profiles require that sole tool using the protocol's unnamed required choice. Explicit caller-named choices remain unsupported, including a caller naming __axOutput directly.

Use muse-image-1.0 through the same chat() method. Text and reference images belong in normal chat content, and generated images arrive in result.results[0].images. Muse Image rejects ordinary function tools. Use transcribe() for Muse Voice batch audio and streaming chat() with PCM16 for realtime transcription; Ax adds no Meta-specific service methods.

Muse Glimmer is self-hosted. Point Ax's vllm, llama-cpp, ollama, or lm-studio profile at the server you operate. Ax does not download, serve, or manage Glimmer weights.

Batch Audio

Use ai.transcribe(...) for batch speech-to-text and ai.speak(...) for batch text-to-speech. These are separate from conversational .chat() audio config.

const transcript = await llm.transcribe({
  audio: { data: base64Wav, format: 'wav' },
  model: 'gpt-4o-mini-transcribe',
  language: 'en',
});

const speech = await llm.speak({
  text: transcript.text,
  model: 'gpt-4o-mini-tts',
  voice: 'alloy',
  format: 'mp3',
});

console.log(transcript.text);
console.log(speech.data);

Gemini's defaults are gemini-3.8-flash-tts for speak() and the dedicated gemini-3.5-transcribe model for transcribe(). Gemini takes no output format, so the returned mime type sets the label: 3.8 TTS returns WAV, while earlier TTS models return raw 24 kHz PCM, reported as pcm16 with its sampleRate and channels.

const gemini = ai({ name: 'google-gemini', apiKey: process.env.GOOGLE_APIKEY! });
const speech = await gemini.speak({ text: 'Hello from Ax.', voice: 'Kore' });
const heard = await gemini.transcribe({
  audio: { data: speech.data, format: speech.format },
});

Providers without the requested audio endpoint throw AxMediaNotSupportedError. Use speech forward options for signature audio artifacts and modelConfig.audio for conversational chat audio.

Common Options

stream (boolean): enable SSE; true by default
thinkingTokenBudget: 'minimal' | 'low' | 'medium' | 'high' | 'highest' | 'none'
showThoughts: include thoughts in output
functionCallMode: 'auto' | 'native' | 'prompt'
debug, logger, tracer, rateLimiter, timeout
timeout, fetch and corsProxy given to one chat() or embed() call override the service's own
timeout is in milliseconds and bounds the wait for the response headers: a request whose response has not started in time fails with AxAIServiceTimeoutError, and a stream that has started runs on

Sampling Parameters

Each provider starts from its own defaults: temperature: 0 for OpenAI Chat, Anthropic and Gemini, and temperature: 0.7 with topP: 1 for openai-responses. Model info lists the sampling parameters a model rejects (notSupported: temperature, topP, topK, presencePenalty, frequencyPenalty):

A provider default for such a parameter is never sent.
An explicit value is sent when the model accepts it for that request, and otherwise dropped with a one-time console.warn naming the setting and the model. Explicit values come from the AI's config, a model key's modelConfig, or the request's modelConfig.
Models whose info sets supported.temperatureOne still take an explicit temperature: 1: every OpenAI model below, and the Anthropic models that deprecated sampling.
GPT-5.1 to 5.4 take sampling while they don't reason, which is their default. GPT-5.5 and 5.6 take it only with reasoning effort none (thinkingTokenBudget: 'none', or reasoningEffort: 'none' in the config). gpt-5, gpt-5-mini, gpt-5-nano and the o-series never take it; the o-series still get maxTokens and n.
A profile without model info (azure-openai, openai-compatible, ...) uses OpenAI's info for an exact o-series model name.
Anthropic: Opus 4.7 and later, Opus 5, Fable 5 and Sonnet 5 deprecated sampling, so they take only temperature: 1 (no topP or topK). The other Claude models take every value with thinking off; while thinking, only temperature: 1, topP of 0.95 or above, and no topK. They reject temperature and topP together (notSupported.temperatureWithTopP): an explicit topP goes alone in place of the default temperature, and an explicit temperature wins over it with a warning. The default temperature: 0 is sent only without thinking, as before. Claude on Vertex keeps that older rule for explicit values too, and warns about a drop.
Gemini: the server-managed Flash models ignore temperature, topP and topK; the Gemini API rejects presencePenalty and frequencyPenalty. Gemini 3 returns one candidate, so n above 1 is dropped (AxGen then gets one sample), and Ax raises a temperature below 1 to 1, as Google recommends, with a one-time warning for an explicit value. Vertex keeps its request shape; presencePenalty is never sent to Gemini and is warned about.

const llm = ai({ name: 'openai', apiKey, config: { model: 'gpt-5.6-luna' } });
// GPT-5.6 takes temperature with reasoning off, so this one is sent.
await gen.forward(llm, values, {
  thinkingTokenBudget: 'none',
  modelConfig: { temperature: 0.2 },
});

A profile without a base URL of its own (openai-compatible, databricks, amazon-bedrock, vertex-ai, ...) needs apiURL: ai(...) throws <Name> requires apiURL without it.

Global Runtime Defaults

Use axGlobals when the app wants one live default for AI requests, generator runs, flows, or metrics:

import { ai, axGlobals, axCreateDefaultColorLogger } from '@ax-llm/ax';
import { metrics, trace } from '@opentelemetry/api';

axGlobals.rateLimiter = async (next, info) => next();
axGlobals.tracer = trace.getTracer('my-app');
axGlobals.meter = metrics.getMeter('my-app');
axGlobals.debug = true;
axGlobals.logger = axCreateDefaultColorLogger();
axGlobals.customLabels = { service: 'api' };
axGlobals.onUsage = (event) => usageQueue.enqueue(event);

const llm = ai({ name: 'openai', apiKey: process.env.OPENAI_APIKEY! });

Rules:

axGlobals.rateLimiter, tracer, meter, logger, debug, abortSignal, and customLabels are live runtime defaults; each operation snapshots them at its start, even if the AI instance already exists.
Precedence is: per-call options, then explicit AI/service options, then current axGlobals, then built-in defaults.
The limiter receives next plus operation, provider, model, streaming state, and previous service usage. It wraps chat and embedding provider execution, including streaming and retries; its errors propagate. It may delay, reject, skip, or invoke next multiple times.
Tracer, meter, and usage-observer failures are fail-open. Limiter failures are fail-closed.
Runtime-hook telemetry contains metadata and usage only, never prompts, outputs, tool arguments, or tool results.
External meter instruments are independent of balancer-local getMetrics() snapshots. Adapt AxMeter to OpenTelemetry at the application boundary; generated packages do not require an OpenTelemetry dependency.
customLabels merge from globals to service to call options; later sources override earlier keys.
abortSignal values are merged, so either a global shutdown signal or a local request signal can cancel the request.
axGlobals.onUsage receives one immutable normalized event for each completed chat or embedding call that reports token usage. A fully consumed stream emits once.
Usage observers are best-effort and fail-open. Ax does not await them; synchronously enqueue events and persist or aggregate them out of band.

Clear process-wide hooks during shutdown or test teardown:

axGlobals.rateLimiter = undefined;
axGlobals.tracer = undefined;
axGlobals.meter = undefined;

Use usageContext for multi-tenant and request attribution:

const llm = ai({
  name: 'openai',
  apiKey: process.env.OPENAI_APIKEY!,
  options: {
    usageContext: {
      tenantId: 'tenant-42',
      feature: 'support-chat',
      attributes: { environment: 'production' },
    },
  },
});

await llm.chat(request, {
  usageContext: {
    userId: user.id,
    requestId: requestId,
    runId: runId,
  },
});

Per-call context overrides service defaults, while attributes are shallow-merged. Events include normalized tokens, provider/model, available session and remote IDs, and a streaming flag. They do not estimate currency cost; calculate that downstream against a versioned pricing table.

DeepSeek Notes

import { ai, AxAIDeepSeekModel } from '@ax-llm/ax';

const deepseek = ai({
  name: 'deepseek',
  apiKey: process.env.DEEPSEEK_APIKEY!,
  config: { model: AxAIDeepSeekModel.DeepSeekV4Flash },
});

DeepSeek's current API models are deepseek-v4-flash and deepseek-v4-pro. Legacy model enum/catalog values remain available for source compatibility, but profile rules are applied only to model IDs verified for the selected deployment.

DeepSeek V4 supports thinking mode. When thinkingTokenBudget is omitted, Ax selects its logical max level and sends thinking: { type: "enabled" } with reasoning_effort: "max". Set thinkingTokenBudget: "none" explicitly to disable it. DeepSeek's API exposes low, medium, high, and max: Ax maps minimal and low to low, preserves medium, maps high to high, and maps highest to max. DeepSeek V4 thinking models support tools, but reject the tool_choice request parameter, so Ax omits auto and Ax-generated __axOutput tool choices for deepseek-v4-pro, deepseek-v4-flash, and deepseek-reasoner while still sending tool definitions. An explicitly forced caller tool choice fails before the request. DeepSeek does not support native JSON schema structured outputs. Ax therefore uses validated json_object for a single required string or code output, including an AxAgent actor's javascriptCode field, without exposing a provider tool. Richer structured outputs use the synthetic __axOutput function when function calling is available, or validated json_object when it is not.

DeepSeek Chat returns thinking traces as reasoning_content. During a tool loop, Ax preserves that field on the assistant tool-call message and sends it back on the following request together with non-null content and the original tool calls. This compatibility mode is declared by the DeepSeek deployment profile; the official OpenAI Chat adapter does not emit or expose reasoning_content.

The same logical default is declared independently for verified DeepSeek V4 rules in the Together, Fireworks, and OpenRouter profiles, then mapped to each deployment's own request dialect. A custom openai-compatible endpoint never inherits it from a DeepSeek-looking model ID.

Other verified deployment rules follow the same policy. Grok 4.6 maps logical max to xhigh, while Grok 4.5 and 4.3 map it to high. Groq GPT-OSS and Cerebras GPT-OSS map it to high; Groq Qwen 3.6 maps it to its documented reasoning-enabled default; Cerebras Gemma 4 and DeepInfra DeepSeek R1 map it to high. An explicit none is sent only for model/deployment combinations that document disabling reasoning. Grok 4.6/4.5 and GPT-OSS on Groq or Cerebras reject none before network I/O because those APIs do not support disabling reasoning for those models.

Hugging Face Router remains conservative: routing policies such as :fastest may choose a different inference provider without changing the base model ID, so Ax does not attach one provider's reasoning contract to that dynamic route.

Extended Thinking

import { ai, AxAIAnthropicModel } from '@ax-llm/ax';

const claude = ai({
  name: 'anthropic',
  apiKey: process.env.ANTHROPIC_APIKEY!,
  config: { model: AxAIAnthropicModel.Claude48Opus },
});

const res = await claude.chat(
  { chatPrompt: [{ role: 'user', content: 'Solve step by step...' }] },
  { thinkingTokenBudget: 'medium', showThoughts: true },
);
console.log(res.results[0]?.thought);
console.log(res.results[0]?.content);

Budget Levels

Level	Anthropic (tokens)	Gemini 2.5 (tokens)	Gemini 3 level
`'none'`	disabled; lowest effort where thinking is always on	0 on Flash/Lite; minimum on Pro	lowest supported, thoughts hidden
`'minimal'`	1,024	200	`minimal`, or `low` when `minimal` is unsupported
`'low'`	5,000	800	`low`
`'medium'`	10,000	5,000	`medium`, or the nearest image/legacy level
`'high'`	20,000	10,000	`high`
`'highest'`	32,000	24,500	`high`

Gemini 3 uses thinkingLevel; Gemini 2.5 and older models use numeric thinkingBudget. Ax selects the wire field after resolving a named model preset to its real model. Gemini 3.8 Flash, Gemini 3.7 Flash, and Gemini 3.1 Pro clamp minimal to low; image and legacy Gemini 3 models clamp to their documented two-level sets. Numeric Gemini 3 budgets fail locally. none always hides returned thoughts, even when the model must still perform its minimum amount of thinking. Gemini 3.8 Live accepts no thinking settings, so Ax sends none. Gemini 3.8 Live Extended Thinking requires a level: Ax sends medium when none is requested and clamps minimal to low.

The native google-gemini deployment profile and its aliases use these Gemini rules, including when configured for Vertex with projectId and region. The separate OpenAI-compatible vertex-ai profile keeps its own request rules and does not inherit native Gemini fields from a Gemini-looking model ID.

For GPT-5.6 and GPT-6 (Astra, Sol, Luna), these map to none, low, low, medium, high, and a top rung that depends on the API surface: xhigh on Chat Completions, which rejects max, and max on the Responses API, which is the only place it is served. These models default an omitted effort to medium, so none is sent explicitly; GPT-6 Astra refuses none and Ax throws. Earlier OpenAI models retain their existing mapping.

Anthropic Model-Specific Behavior

Opus 5.5, Fable 5.1, Fable 5, Opus 5, Opus 4.8, 4.7, and 4.6 plus Sonnet 5: adaptive thinking, no manual budget_tokens, and no temperature / topP / topK. When thoughts are requested, Ax asks Anthropic for summarized display; when they are hidden, Ax explicitly requests display: 'omitted'.
Opus 5.5, Fable 5.1, and Fable 5 always think, so thinkingTokenBudget: 'none' sends the lowest effort with thoughts hidden instead of disabling it.
Opus 5 and Sonnet 5 think by default, so 'none' sends thinking: { type: 'disabled' }. Opus 5 only allows that at effort 'high' or below, so Ax rejects 'none' combined with 'xhigh' or 'max'.
Opus 5.5 and Fable 5.1 refuse forced tool choice: Ax throws for functionCall: 'required' or a named function, and structured output uses the native output_config.format path.
Opus 4.8, Opus 5, Opus 5.5, Fable 5, and Fable 5.1 keep a later system message in place on the first-party API; other models hoist it into the system prompt.
Opus 4.5: budget_tokens + effort levels (capped at 'high')
Other thinking models: budget tokens only

Anthropic modelConfig.effort can be set directly on a request. Fast mode and task budgets are Anthropic-only opt-ins; taskBudget.total must be at least 20,000 tokens.

const res = await claude.chat({
  chatPrompt: [{ role: 'user', content: 'Review this migration plan.' }],
  modelConfig: {
    effort: 'xhigh',
    speed: 'fast',
    taskBudget: { type: 'tokens', total: 64_000 },
  },
});

Custom Thinking Levels

const claude = ai({
  name: 'anthropic',
  apiKey: '...',
  config: {
    model: AxAIAnthropicModel.Claude48Opus,
    thinkingTokenBudgetLevels: {
      minimal: 2048,
      low: 8000,
      medium: 16000,
      high: 25000,
      highest: 40000,
    },
    effortLevelMapping: {
      minimal: 'low',
      low: 'medium',
      medium: 'high',
      high: 'high',
      highest: 'max',
    },
  },
});

Embeddings

const { embeddings } = await llm.embed({
  texts: ['hello', 'world'],
  embedModel: 'text-embedding-005',
});

Vertex AI Locations

When projectId and region are set for Google Gemini or Anthropic on Vertex AI, Ax selects the service hostname from the location automatically:

global uses aiplatform.googleapis.com
us and eu use the multi-region .rep.googleapis.com endpoints
regional locations such as us-central1 use {region}-aiplatform.googleapis.com

Pass the canonical lower-case Vertex location ID. Ax preserves the supplied value and does not normalize or validate it.

gemini-embedding-2 is the exception: Vertex serves it only at global through :embedContent, so Ax sends its embeddings there whatever region is set. Each request embeds exactly one text (pass several and embed() throws, because Vertex would fuse them into one vector), and no task type is sent even when embedType is configured, since Vertex ignores one for this model. Put task instructions in the text itself, e.g. task: search result | query: {content}. Other embedding models keep the regional :predict endpoint.

The generated Python, Java, C++, Go, and Rust clients accept the same projectId / project_id, region, and optional endpointId / endpoint_id options. In generated clients, apiKey / api_key is a caller-supplied bearer access token (or GOOGLE_VERTEX_ACCESS_TOKEN); ADC discovery and automatic token refresh remain host-owned. An explicit baseUrl / base_url always wins.

Context Caching

const result = await gen.forward(llm, { code, language }, {
  mem,
  sessionId: 'code-review-session',
  contextCache: {
    ttlSeconds: 3600,
    cacheBreakpoint: 'after-examples',
  },
});

Breakpoint values: 'system' | 'after-functions' | 'after-examples'

Provider behavior:

Google Gemini: explicit caching with cache resource ID and auto TTL refresh; failed refreshes recreate or fall back uncached, and rejected Ax-managed caches retry once without the cache
Anthropic: implicit via cache_control markers
Meta Responses and Chat Completions cache automatically and accept an optional promptCacheKey; Responses also accepts promptCacheRetention: 'in_memory' | '24h'. Meta Messages has no cache marker or cache-key field, so Ax strips generic cache_control annotations on that profile.
OpenAI: explicit prompt_cache_breakpoint markers, GPT-5.6+ only; GPT-6 also gets ttl: '30m' in prompt_cache_options. Earlier families cache automatically and predate the parameters, so nothing is sent to them. Only the openai provider opts in — Azure OpenAI shares the request builder and the same model enum, so a gpt-5.6-* deployment sends nothing, and openai-responses sends breakpoints for GPT-6 only (it does report cacheCreationTokens, which is provider-wide)

OpenAI prompt cache keys

GPT-5.6+ needs a key that is stable per conversation to match reliably; it routes the request to the shard the cache lives on. Set promptCacheKey, or let it fall back to sessionId. Keep it under roughly 15 requests/minute per key.

Every OpenAI Responses request sends prompt_cache_key, with or without caching: the promptCacheKey, else the sessionId, the call's before the service's. Chat Completions sends it only with GPT-5.6+ caching.

const result = await gen.forward(llm, values, {
  mem,
  promptCacheKey: `review:${pullRequestId}`,
  contextCache: {},
});

AxGen forwards these provider options after merging program defaults with the per-call options. Generated language packages preserve the same promptCacheKey / sessionId / contextCache forwarding contract.

Markers must not move. A breakpoint marker is part of its content block, so marking only "the newest stable message" each turn un-marks what the previous turn marked, changing the prefix and voiding the entry that turn wrote. Ax marks by absolute index from the front, which is stable for an append-only conversation. Anything that rewrites the front of the history — dynamically added functions changing the system prompt, or mem.rewindToTag — costs a cache miss.

Keep caching on for the whole conversation. The provider marks all or nothing, so a turn that omits contextCache sends the prompt unmarked and the next turn rewrites the cache from scratch.

External Registry (serverless)

const accountId = getRequiredAccountId();
const registry: AxContextCacheRegistry = {
  get: async (key) => {
    const value = await redis.get(`context-cache:${accountId}:${key}`);
    return value ? JSON.parse(value) : undefined;
  },
  set: async (key, entry) => {
    const ttl = Math.max(1, Math.ceil((entry.expiresAt - Date.now()) / 1000));
    await redis.set(
      `context-cache:${accountId}:${key}`,
      JSON.stringify(entry),
      { ex: ttl }
    );
  },
};

Ax registry keys are content-based and are not account-scoped. Require a stable tenant/account namespace when cross-account cache sharing is unsafe; do not silently fall back to a global namespace.

AWS Bedrock

Use the amazon-bedrock profile for Bedrock's OpenAI-compatible Mantle endpoint. Supply the account/region-specific OpenAI base URL and a Bedrock API key explicitly:

const bedrock = ai({
  name: 'amazon-bedrock',
  apiURL: process.env.BEDROCK_OPENAI_BASE_URL!,
  apiKey: process.env.BEDROCK_API_KEY!,
  config: { model: process.env.BEDROCK_MODEL_ID! },
});

To reach OpenAI's GPT models on those endpoints through the openai or openai-responses provider, name them with Bedrock's IDs: openai.gpt-6-sol on bedrock-mantle, or a cross-Region inference profile such as us.openai.gpt-6-sol or global.openai.gpt-6-astra on bedrock-runtime. Ax applies the named model's contracts: its built-in model info (so temperature and top_p are not sent), the GPT-5.6 and GPT-6 effort ladders, Astra's rules, and, on openai, routing the GPT-6 family to <apiURL>/responses. It leaves off what Bedrock does not serve: prompt-cache breakpoints on Chat Completions (Bedrock caches these models on the Responses API only) and Astra chat sessions, since async tools, steering, and reasoning updates are not available there.

const gpt = ai({
  name: 'openai',
  apiURL: 'https://bedrock-runtime.us-east-1.amazonaws.com/openai/v1',
  apiKey: process.env.BEDROCK_API_KEY!,
  config: { model: 'us.openai.gpt-6-astra' as AxAIOpenAIModel },
});

The separate AWS package remains available when native AWS SDK authentication, regional fallback, or non-Mantle Bedrock behavior is required:

import { AxAIBedrock, AxAIBedrockModel } from '@ax-llm/ax-ai-aws-bedrock';

const bedrock = new AxAIBedrock({
  region: 'us-east-2',
  fallbackRegions: ['us-west-2'],
  config: { model: AxAIBedrockModel.ClaudeSonnet5 },
});

The native client uses Bedrock Converse and ConverseStream. Claude models advertise native functions, streaming, image/document input, prompt-cache breakpoints, and their verified thinking modes. GPT-6 Sol, Luna, and Astra (AxAIBedrockModel.Gpt6Sol, Gpt6Luna, Gpt6Astra) advertise native functions, streaming, image input, and reasoning effort. Structured-output and service-tier support remain model-specific; query bedrock.getFeatures(model) instead of assuming every Bedrock model has the same capabilities. The AWS SDK remains a dependency of @ax-llm/ax-ai-aws-bedrock only and is not pulled into @ax-llm/ax.

Use contextCache.ttlSeconds for a 5-minute or supported 1-hour cache point, and thinkingTokenBudget for legacy budget thinking or the corresponding adaptive-thinking effort on current Claude models:

const response = await bedrock.chat(
  {
    chatPrompt: [
      { role: 'system', content: 'Use the supplied policy.', cache: true },
      { role: 'user', content: 'Summarize the policy.' },
    ],
  },
  {
    stream: true,
    contextCache: { ttlSeconds: 3600 },
    thinkingTokenBudget: 'medium',
  }
);

On the native Bedrock client, Claude Sonnet 5 and Opus 5.5 always use adaptive thinking and reject thinkingTokenBudget: 'none'; Claude Opus 5 permits disabling it. On GPT-6, thinkingTokenBudget sets the reasoning effort (minimal → low, highest → max, none turns reasoning off), except that Astra rejects 'none' before sending. GPT-6 requests omit temperature and topP, which these models do not accept.

Vercel AI SDK Integration

import { generateText } from 'ai';
import { ai } from '@ax-llm/ax';
import { AxAIProvider } from '@ax-llm/ax-ai-sdk-provider';

const axAI = ai({
  name: 'openai',
  apiKey: process.env.OPENAI_APIKEY ?? '',
});
const model = new AxAIProvider(axAI);

const result = await generateText({
  model,
  prompt: 'Hello!',
});

MCP + AxJSRuntime

import { AxMCPClient } from '@ax-llm/ax';
import { axCreateMCPStdioTransport } from '@ax-llm/ax-tools';

const transport = axCreateMCPStdioTransport({
  command: 'npx',
  args: ['-y', '@anthropic/mcp-server-filesystem'],
});
const client = new AxMCPClient(transport);

For server notifications, call client.startListening({ signal, onError }) or attach the client through AxMCPEventSource. The event adapter is preferred for autonomous work because protocol callbacks only enqueue; explicit routes decide whether to observe, invalidate, resume, or wake.

For signed UCP lifecycle requests, mount AxUCPWebhookEventSource.ingest(request) in application-owned HTTP hosting. Signature, profile, digest, freshness, and replay verification completes before the event runtime sees the request.

Critical Rules

Use ai() factory for all providers.
Use axAIProfiles() as the source of truth for names. Core names include 'openai', 'openai-compatible', 'openai-responses', 'anthropic', 'google-gemini', 'azure-openai', 'deepseek', 'meta', 'meta-chat', 'meta-messages', 'mistral', 'cohere', 'grok', routers such as 'together', 'openrouter', and 'orcarouter', hosted inference profiles, and configurable local runtimes.
Thinking constraints on Anthropic: every adaptive-thinking model omits temperature, topP, and topK; older thinking models ignore temperature and topK, with topP only sent if >= 0.95.
ai({ name: 'amazon-bedrock', apiURL: ... }) targets Bedrock's OpenAI-compatible endpoint. new AxAIBedrock() remains the separate AWS-native runtime client and keeps the AWS SDK out of core.
Vercel AI SDK uses AxAIProvider wrapper.

Examples

Fetch these for full working code:

Embeddings — embedding generation
Anthropic Thinking — extended thinking with functions
Anthropic Thinking Separation — thinking separation
Anthropic Web Search — Anthropic web search
OpenAI Web Search — OpenAI web search
OpenAI Responses — OpenAI responses API
o3 Reasoning — o3 reasoning
Gemini Context Cache — Gemini context caching
Gemini Files — Gemini file handling
Grok Live Search — Grok live search
OpenAI-Compatible — custom OpenAI-compatible base URL
Meta Muse — Muse Spark and Muse Image through chat()
Meta Voice — Muse Voice batch transcription
Vertex AI Auth — Vertex AI authentication
MCP Stdio — MCP stdio transport
MCP HTTP — MCP HTTP transport
Telemetry — OpenTelemetry tracing
Multi-Modal — image handling

Do Not Generate

Do not use new AxAIOpenAI(...) or similar class constructors for standard providers; use ai().
Do not hardcode provider class names when ai({ name: ... }) covers the provider.
Do not mix thinkingTokenBudget with explicit temperature on Anthropic thinking models.
Do not confuse the amazon-bedrock OpenAI-compatible profile with the separate AWS-native AxAIBedrock client.
Do not invent Meta image, file, model-listing, or response-lifecycle methods. Muse Image uses chat(), Voice uses transcribe() or realtime chat(), and inline data/URLs use normal chat content.
Do not omit resourceName and deploymentName for Azure OpenAI.

GPT-6 Astra and automatic sessions (TypeScript)

Select ai({ name: 'openai', config: { model: AxAIOpenAIModel.GPT6Astra }, apiKey }). Import ai and AxAIOpenAIModel from @ax-llm/ax. Ax automatically routes Astra through Responses. Existing defaults are unchanged. Use thinkingTokenBudget: 'low' and serviceTier: 'standard'. Astra requires reasoning; minimal maps to low and none throws. Unsupported sampling and log-probability options are removed. EU residency does not support priority processing.

GPT-6 Sol (AxAIOpenAIModel.GPT6Sol) and Luna (AxAIOpenAIModel.GPT6Luna) also route through Responses, since Chat Completions refuses their function tools while they reason. Unlike Astra they accept thinkingTokenBudget: 'none', and chat sessions and configuration_update remain Astra-only. OpenAI reports a GPT-6 priority request's served tier as fast; Ax records it as priority.

Keep calling forward() and streamingForward(). Declare independent tools with fn('lookup').description('...').execution('background').handler(...).build(). Ordinary tools default to blocking. JavaScript promises and MCP annotations do not opt a tool into background execution. Set asyncMode: 'off' to use the ordinary tool loop. Providers without session support retain that loop.

Use const control = runControl() and pass { control } in forward options. Call control.steer(text), control.setThinkingTokenBudget('high'), or control.abort(). control.onEvent(listener) observes queued/applied updates, run lifecycle, tool activity, and model output activity. A run emits started, then completed or failed (with its error). control.abort() emits aborted at once, and the run then ends as failed with the abort error. A consumer that stops a streamingForward early (for example with break) ends the run as aborted instead. Untargeted updates apply to the root and future descendants. { target: 'root/nodeName' } restricts an update to a flow node and its descendants. Completed nodes are not rerun. Controller-attached runs bypass result caching; provider prompt caching remains enabled.

HTTP streaming needs no WebSocket dependency. With a configured host options.webSocket, steering can apply natively during generation; otherwise it applies at the next response boundary. Observe the applied event's timing. An update queued while a request is in flight applies when the next step starts. If that request gave the final answer, the run takes one more step to apply it, and the answer comes from that step; a steer stays in the conversation for the steps after it. Reasoning updates use continuation input items, retaining the original prefix. Steering that awaits tool input is continued even when its pending notification arrives after completion. Duplicate acknowledgements do not apply an update twice.

Ax owns tool execution and result submission. Only completed calls execute; pending results are incorporated before successful final output. Streaming clients reset accumulated output when version changes; provisional answers never count as successful completion. Sessions pin the selected provider and model and do not reconnect or replay calls after a failure. Cancellation requests tool cancellation; it does not undo external work. The native Responses wire client is internal. Custom providers may implement the optional normalized openChatSession contract; existing .chat() services work. Session adapters should expose their transport abort signal so pending host work does not prevent cancellation or disconnection from ending the run. Sessions preserve provider defaults and model-alias settings; explicit request settings win.

Runnable examples: typescript/generation/astra.ts, astra-async-tools.ts, astra-steering.ts, astra-reasoning-update.ts, astra-session-lifecycle.ts, and typescript/short-agents/astra-background.ts. Generated-language session support is partially implemented in Python, Go, Java, C++, and Rust. Their language galleries include provider-backed examples, but full parity remains open in the AxIR backlog. Generated flow groups now dispatch owned workers, with a traced serial fallback for custom implementations without worker factories. Remaining MCP invocation and session acceptance evidence is tracked in the backlog. See docs/COMPILER.md for the implementation status.

Files through provider routing

Provider routing preserves native file items when the selected provider and model support files, including filename, MIME type, cache flags, and extraction metadata. Supplying extracted text does not replace a file that the provider can consume natively. For unsupported providers, extracted text or a configured file-to-text callback supplies text; fallback policy can degrade, skip, or reject the file. The original conversation retains the file for later turns. Generated Python, Go, Java, C++, and Rust routers apply this policy in shared Core after selection.

ax-ai

AI Provider Codegen Rules (@ax-llm/ax)

Quick Setup