LiteLLM

Overview

LiteLLM provides a single API to call 100+ LLM providers — OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Azure, Bedrock, Ollama, and more. Write your code once using the OpenAI SDK format, then switch providers by changing a model string. As a proxy server, it adds load balancing, fallbacks, rate limiting, spend tracking, and API key management for teams.

When to Use

• Using multiple LLM providers and want a unified interface
• Need automatic fallbacks (if Claude is down, use GPT)
• Cost tracking across multiple providers and teams
• Load balancing requests across multiple API keys or models
• Self-hosted proxy to manage LLM access for a team

Instructions

Setup