Routing Architecture Guide

Overview

LiteLLM-RS implements a high-performance, lock-free routing system with 7 strategies for intelligent provider selection across 66+ LLM providers.

Key Design Principles

• Lock-free: Uses DashMap for concurrent access without mutex locks
• Health-aware: Continuous provider health monitoring
• Configurable: Multiple strategies with runtime selection
• Fallback chains: Automatic failover to backup providers

Routing Strategies

1. SimpleShuffle (Default)