Why host a LLM on your own VPS in 2026
Running a language model on your own server offers four concrete advantages. First, privacy: your data never leaves the server, making the setup natively GDPR-compliant without complex data processing agreements. Second, cost: no API tokens to pay for, just a fixed, predictable VPS bill. Third, guaranteed availability: no dependency on a third-party service, no quotas, no external outages. Finally, customization: you choose the model, the quantization level, the system parameters, and you can fine-tune or switch models with a single command.
What Ollama enables you to do
- Local inference — Llama 3.2, Qwen2.5, Phi-3-mini and over 150 ready-to-use GGUF models
- OpenAI-compatible API — connect n8n, Open WebUI or any existing client without changing a single line of code
- Multimodality — LLaVA and Gemma3 for local image analysis and description
- Built-in interactive agent — v0.32.0: the Ollama terminal becomes an agent capable of coding, searching, and delegating sub-tasks
- CPU and GPU modes — AVX-512 on EPYC or Xeon: 8-12 tok/s without a graphics card on an 8 GB RAM VPS
Realistic requirements for running a LLM on a VPS
For Phi-3-mini or Llama 3.2 3B in Q4_K_M quantization, plan for a minimum of 4 vCPUs, 8 GB of RAM and 20 GB of NVMe storage. For Qwen2.5-7B, upgrade to 16 GB of RAM. The recommended operating system is Ubuntu 24.04 LTS, which includes all required dependencies in its repositories. No GPU is needed for 3B to 7B models in Q4_K_M: the AVX-512 instruction sets present on recent EPYC and Xeon processors are sufficient. Port 11434 (Ollama API) should only be exposed on localhost; remote access must always go through a reverse proxy.
Deploy Ollama on a VPS in 5 steps
Install Ollama
Connect via SSH to your VPS and run the official script: curl -fsSL https://ollama.com/install.sh | sh. The ollama systemd service starts automatically and listens on 127.0.0.1:11434.
Download your first model
Pull Phi-3-mini (2.2 GB, ideal for getting started): ollama pull phi3:mini. For a more capable model, use ollama pull llama3.2:3b or ollama pull qwen2.5:7b depending on your available RAM.
Test from the command line
Launch an interactive session: ollama run phi3:mini. Type your question and press Enter. To query the REST API: curl http://localhost:11434/api/generate -d '{"model":"phi3:mini","prompt":"Hello"}'.
Expose the API over HTTPS with Nginx
Install Nginx and Certbot, create a vhost for your-domain.com with proxy_pass http://127.0.0.1:11434;, obtain a Let's Encrypt certificate with certbot --nginx -d your-domain.com, then add HTTP Basic authentication or a Bearer header to protect access.
Connect Open WebUI
Deploy Open WebUI (see the dedicated article) and enter your secured Ollama API URL. ollama serve remains active as a systemd service: no manual command is needed after a restart.
By default, Ollama listens on OLLAMA_HOST=127.0.0.1, which blocks all external access. Never change this variable to listen on 0.0.0.0 without protection: an open port 11434 on the internet gives full access to the model and files reachable via the API. Always use a Nginx or Caddy reverse proxy with authentication (HTTP Basic or Bearer token in the Authorization header). Regularly verify that the port is not exposed: ss -tlnp | grep 11434.
Going further: building a complete AI stack
Ollama alone is an inference engine. To build a complete, self-hosted AI stack, pair it with Open WebUI (a multi-model chat interface with conversation and user management) and Dify for orchestrating complex AI workflows. Add Gatus to monitor the availability of each service with automatic alerts. Our dedicated guides cover each component: deploying Open WebUI on a VPS, configuring Dify, and setting up Gatus for monitoring. By combining these three tools with Ollama, you get a sovereign AI platform with no cloud vendor dependency, at a fixed monthly cost.
