Ollama Setup

Overview

Auto-configure Ollama for local LLM deployment, eliminating hosted API costs and enabling offline AI inference. This skill handles system assessment, model selection based on available hardware (RAM, GPU), installation across macOS/Linux/Docker, and integration with Python, Node.js, and REST API clients.

Prerequisites

• macOS 12+, Linux (Ubuntu 20.04+, Fedora 36+), or Docker runtime
• Minimum 8 GB RAM for 7B parameter models; 16 GB for 13B models; 32 GB+ for 70B models
• Optional: NVIDIA GPU with CUDA drivers for accelerated inference (nvidia-smi to verify)
• Optional: Apple Silicon (M1/M2/M3) for Metal-accelerated inference on macOS
• Disk space: 4-40 GB depending on model size (quantized weights)
• Package manager: brew (macOS), curl (Linux), or docker (containerized)

Instructions

1. Detect the host operating system and available hardware using uname -s, free -h (Linux) or vm_stat (macOS), and nvidia-smi (if GPU present)
2. Select appropriate models based on available RAM:
   • 8 GB: llama3.2:7b (4 GB), mistral:7b (4 GB), phi3:14b (8 GB)