LangChain Reference Architecture (Python)

Overview

Eight months into a LangChain service, a code review surfaces the mess. Twelve chain definitions live inlined inside FastAPI route handlers. Three retrievers are constructed at module-global scope, one bound to tenant_id="acme" because that was the first tenant in the pilot — that retriever now returns Acme's documents to every other tenant, a P33 leak that has been live in production for six weeks. max_retries=6 is hardcoded at four separate call sites. A RunnableWithMessageHistory backed by the default InMemoryChatMessageHistory loses every conversation on pod restart (P22) — which is most days, because Cloud Run scales to zero. Config is read from os.environ in three modules with three different fallback strategies. There is no place to put a new provider without touching seven files, and nobody remembers why the retriever is built at import time.