Modal Endpoints

Use this skill when deploying Auto Endpoints: managed, OpenAI API-compatible LLM inference on Modal with minimal configuration (SDK/CLI 1.5.1+).

Endpoints are built on Modal Servers. For custom server code, see modal-servers. For hand-rolled vLLM/SGLang Functions, see modal-web-serving / modal-scaling-performance.

When to Use This Skill

• Need a production LLM API with one CLI command
• Serving catalog models (Qwen, Kimi, Gemma, DeepSeek, Nemotron, GPT-OSS, GLM, …)
• Serving fine-tunes from Hugging Face or a Modal Volume
• Want scale-to-zero, usage-based GPU billing, and tuned engine recipes
• Prefer managed speculative decoding / recipes over writing server code first

Prefer a custom @app.server when you need full control of engine flags, multi-model routers, or non-OpenAI protocols.

Quick Start