Quarkus AI — LangChain4j Integration

Overview

Expert guidance for integrating Large Language Models into Quarkus 3.36.0 applications using Quarkus LangChain4j. Provides declarative AI services via @RegisterAiService, tool calling, RAG pipelines, chat memory, streaming, and production observability.

Key Capabilities:

• Declarative AI service interfaces with @RegisterAiService
• @SystemMessage and @UserMessage for prompt engineering
• Tool calling with @Tool-annotated CDI beans and @ToolBox
• RAG with EmbeddingStore, EmbeddingStoreContentRetriever, RetrievalAugmentor
• Chat memory with MessageWindowChatMemory and @MemorySessionId
• Streaming responses with Multi<String>
• Structured output (JSON to POJO mapping)
• 20+ supported LLM providers (OpenAI, Anthropic, Ollama, Mistral, Bedrock…)
• Vector stores: pgvector, Redis, Chroma, MongoDB Atlas, Pinecone, Qdrant
• Observability: Micrometer metrics + OpenTelemetry tracing
• MCP (Model Context Protocol) client support
• Guard rails and content moderation