NLWeb LLM & Embedding Providers

Before writing code

Fetch live docs:

1. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/docs/nlweb-providers.md for the canonical provider list and config schema.
2. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/config/config_llm.yaml for the exact model IDs and env-var names currently shipped.
3. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/config/config_embedding.yaml for embedding defaults.
4. Inspect AskAgent/python/llm_providers/<provider>.py for the SDK calls the provider class makes.
5. Web-search the latest release notes — new providers and models get added often.

Conceptual Architecture

Mixed-Mode = Many Small LLM Calls

NLWeb's pipeline doesn't make one big LLM call per query. It makes many small calls: decontextualize the query, detect Schema.org item type, route to a tool, rank results, optionally summarize/generate. Each call has a strict <returnStruc> JSON schema in prompts.xml. Cost and latency are dominated by the number of calls, not the size of any single one.

High / Low Tier Model Selection