Why self-host Ollama on a VPS
Ollama is not an interface but an inference engine: it downloads quantized models and serves them via a local REST API, with an OpenAI-compatible endpoint (/v1/chat/completions). The benefit of a dedicated VPS is twofold. First, cost: a fixed-price VPS replaces an API bill that explodes with volume. Then, privacy: your prompts never leave the server, which is decisive for handling regulated data. Ollama becomes the shared LLM backend of your entire stack, which you plug into AnythingLLM, Flowise, LibreChat or your own scripts, simply by replacing the base URL with that of your VPS.
The concrete benefits of a self-hosted Ollama
- Fixed inference cost: no more billing per token, regardless of the volume of requests.
- OpenAI-compatible API: you replace the base URL without rewriting your client code.
- Absolute privacy, prompts and responses stay on your infrastructure.
- Free choice and switching between dozens of models via
ollama pull. - Single LLM backend shared across all your self-hosted AI tools.
- Possible offline operation, without dependency on an external service.
Hardware and software requirements
Ollama is the most RAM-demanding on this list, because the model must fit in memory. A quantized 7B model (Q4) requires about 5 to 6 GB of RAM; so count on at least 8 GB of RAM to comfortably serve a 7B/8B on CPU, and 16 GB for larger models or several loaded models. On pure CPU, generation remains slow: for responsive production, a VPS with a GPU radically changes things. Plan for 20 to 40 GB of disk depending on the number of models downloaded. Docker recommended, plus a subdomain (e.g. api-ia.your-domain.com) and port 443.
Deploy Ollama with Docker and a secured API
Install Ollama in a container
Run docker run -d -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 --name ollama ollama/ollama. The binding on 127.0.0.1 is deliberate: the API must never be exposed directly, because it is unauthenticated by default.
Download a model
Retrieve a model with docker exec -it ollama ollama pull llama3.1:8b (or mistral, qwen2.5). Test locally: docker exec -it ollama ollama run llama3.1:8b 'Bonjour'.
Verify the local API
Call the endpoint: curl http://127.0.0.1:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"test"}'. Confirm that the response arrives as a stream.
Add an authentication layer at the proxy
Since Ollama has no native auth, place a reverse proxy in front. With Caddy, expose api-ia.your-domain.com, add a token check (Authorization header) via a directive, then reverse_proxy localhost:11434. Let's Encrypt handles the TLS.
Lock down the firewall
With UFW, only allow 443 (and 22) inbound, and explicitly block external 11434. Check from outside that https://api-ia.your-domain.com does require the token.
Connect your applications
In your clients (OpenAI SDK, Flowise, LibreChat), point the base URL to https://api-ia.your-domain.com/v1 and pass your token. Your existing calls work without any other modification.
On a CPU VPS, the first request after a period of inactivity is slow because the model is reloaded into memory. Keep it resident by setting OLLAMA_KEEP_ALIVE=-1, and limit the number of concurrent models with OLLAMA_MAX_LOADED_MODELS=1 to avoid saturating the RAM. Favor Q4_K_M quantizations: the best speed/quality compromise on a processor.
The official documentation
For advanced configuration and options specific to the tool, refer to the official Ollama documentation. This guide covers going live on a VPS; the vendor documentation remains the reference for fine tuning, major updates and specific use cases.
