LiteLLM Streaming & Tool Calling

Source: https://docs.litellm.ai/stream, https://docs.litellm.ai/docs/completion/function_call

Purpose

Unified streaming and tool/function calling across all supported LLM providers. LiteLLM normalizes provider differences behind an OpenAI-compatible API: stream=True for sync/async streaming, stream_options={"include_usage": True} for token usage in streams, tools + tool_choice for function calling, and runtime capability checks (supports_function_calling, supports_parallel_function_calling).

When to Use

• Stream LLM responses to users for real-time UX (sync or async)
• Implement tool calling that works across OpenAI, Anthropic, Azure, Grok, and other providers
• Check model capabilities at runtime before sending tools
• Handle parallel function calls (multiple tool invocations per response)
• Convert Python functions to tool schemas automatically with function_to_dict()
• Enable tool calling on models without native support via add_function_to_prompt

Basic Usage