OpenRouter Reference Architecture

Overview

OpenRouter serves as a unified LLM gateway, abstracting provider complexity. A production architecture wraps it with caching, rate limiting, cost controls, observability, and async processing. This skill provides three reference architectures: simple (single service), standard (microservice), and enterprise (event-driven).

Prerequisites

• An OpenRouter API key (sk-or-v1-...) exported as OPENROUTER_API_KEY — see the openrouter-install-auth skill for setup
• Python 3.8+ with the OpenAI SDK; FastAPI + Pydantic for Architecture 2's AI service, and a Redis instance (with the redis package) for Architecture 2's cache and Architecture 3's queue/results store
• SQLite or Postgres if you implement Architecture 2's budget enforcer
• Your scale numbers — team size, requests/day, and latency needs drive the decision in Choosing an Architecture

Instructions

1. Score your system against the Choosing an Architecture table: team size, requests/day, latency needs, budget-tracking granularity, failure handling, observability.
2. Start with Architecture 1 (Simple): one shared client (max_retries=3, timeout=30.0) behind the logging complete() wrapper.
3. When you need task routing, caching, and per-user budgets, move to Architecture 2 (Standard): a FastAPI /v1/complete endpoint with the ROUTING_TABLE, cache-first lookup, budget check, and a fallback chain (models + route: "fallback").
4. At 100K+ requests/day or mixed sync/async workloads, adopt Architecture 3 (Enterprise): queue (Redis/SQS) → auto-scaling workers running worker_loop() → results store, with OTEL metrics feeding dashboards and alerts.
5. Whichever tier you land on, route every call through the same OpenRouter client wrapper per Enterprise Considerations — consistent logging, cost tracking, and no budget bypass.