Ollama

Overview

Ollama runs open-weight language models on your own hardware: ollama run gemma4 downloads a model and opens a chat. A local server on http://localhost:11434 serves a native API (/api/*) plus OpenAI-compatible (/v1/chat/completions, /v1/responses, /v1/embeddings) and Anthropic-compatible (/v1/messages) endpoints, with no API key and no per-token cost. This skill covers installation, model management, API integration from Node.js and Python, structured output, custom models, and context/GPU configuration.

Instructions

Step 1: Installation

# macOS (Homebrew formula: CLI + server; `brew services start ollama` runs it in the background)
brew install ollama

# Docker, CPU only
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
# Docker with NVIDIA GPUs (needs the NVIDIA Container Toolkit on the host)
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama