Ollama Proxy — One Endpoint for All Your Ollama Instances

You have Ollama running on multiple machines. Instead of hardcoding IPs and manually picking which Ollama instance to hit, point everything at the Ollama proxy. The Ollama proxy routes to the best available device automatically.

Before:  App → http://macmini:11434  (one Ollama instance, hope it's not busy)
After:   App → http://ollama-proxy:11435   (Ollama proxy picks the best machine)

Set up the Ollama proxy

pip install ollama-herd    # PyPI: https://pypi.org/project/ollama-herd/

On one machine (the Ollama proxy):

herd    # starts the Ollama proxy on port 11435