Ollama

About Ollama

What gets installed

Ready when the VPS starts

Compatible OS

Ubuntu 24.04

Application

Ollama

Tech stack

DockerGollama.cpp

Minimum RAM

8 GB minimum (16 GB recommended)

Minimum CPU

2 vCPU

Default port

11434

Access

root over SSH

Key features

• OpenAI-compatible API — swap the base URL and your existing code works instantly
• One-command model management: ollama pull llama3.1:8b, ollama list, ollama rm
• Supports Llama 3, Mistral, Qwen, DeepSeek, Gemma, Phi and 100+ other models
• CPU-only mode works out of the box; optional GPU passthrough for faster inference
• Multi-model server: load several models concurrently, each on its own port or context
• Persistent model storage in a Docker volume — models survive container restarts

When to use this solution?

Guide

Deploy Ollama on your VPS

1. 1

   Create the VPS

   Order a ServOrbit VPS with at least 8 GB of RAM for a 7 to 8 billion parameter model quantised to Q4 on CPU, 16 GB for larger models. The system installed is Ubuntu 24.04. For production-grade response times, a server fitted with a GPU changes the experience markedly.

2. 2

   Deploy Ollama

   The installation, when the VPS is created or later from your client area, creates the ollama/ollama container, mounts the ollama volume on /root/.ollama so your models survive reboots, and publishes the service on the loopback interface, at 127.0.0.1:11434. Nothing is exposed to the internet at this stage: that is deliberate, since Ollama has no authentication of any kind.

3. 3

   Connect for the first time

   Ollama has no sign-in screen: it is an HTTP API paired with a command line. Over SSH on the VPS, run docker exec -it ollama ollama pull llama3 then docker exec -it ollama ollama run llama3. Then plug in an interface — Open WebUI for instance — rather than exposing the API as it stands.

4. 4

   Pull your first model

   Fetch a model with docker exec -it ollama ollama pull llama3.1:8b, check that it is registered with docker exec -it ollama ollama list, then test from the VPS: curl http://127.0.0.1:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"Hello"}'.

5. 5

   Account for the lack of authentication

   Ollama asks for no credentials and knows nothing of API keys: anyone who reaches the service can query your models and pull new ones. That is why ServOrbit attaches no domain to it — a vhost's encryption would protect the transport, it would identify nobody. Two usages hold up: work through an SSH tunnel, or put an authenticating gateway in front of Ollama — LiteLLM for instance — and publish only that.

6. 6

   Connect your tools

   In Open WebUI, open Settings then Connections and enter http://localhost:11434. From an OpenAI SDK running on the VPS itself, set base_url='http://127.0.0.1:11434/v1' and api_key='ollama'. From any other machine there is no public entry point: open an SSH tunnel with ssh -L 11434:127.0.0.1:11434 root@<your-vps-ip> then point at http://localhost:11434/v1. The port may be reassigned at install time: use the one shown on the app's card in your client area, 11434 being only the catalogue value.