Why self-host LocalAI on a VPS
Where Ollama focuses on text, LocalAI aims for broad compatibility with the OpenAI API: it exposes /v1/chat/completions, /v1/embeddings, /v1/images/generations and even audio transcription, on a single API. For a team that already has code written against the OpenAI SDK, LocalAI is a near-transparent replacement: you change the base URL and the key, the rest works. On a VPS, you get this versatility without external dependency or per-call cost. This is especially relevant when you need text generation, vectors for RAG and, occasionally, images all at once, without multiplying providers and keys.
The concrete benefits of a self-hosted LocalAI
- Drop-in replacement for the OpenAI API: no rewriting of the client code.
- A single API for chat, embeddings, images and audio.
- Support for multiple backends (llama.cpp, diffusers, whisper) under a unified interface.
- Open model formats (GGUF) that are downloadable and interchangeable.
- No usage-based billing: a fixed VPS cost for all types of generation.
- Data and generations kept entirely on your server.
Hardware and software requirements
Since LocalAI is versatile, its needs depend on the functions enabled. For 7B chat and embeddings on CPU, aim for 8 GB of RAM and 4 vCPU. If you enable image generation (Stable Diffusion) or audio, the load rises sharply and a VPS with a GPU becomes strongly recommended. Disk is a key point: GGUF and diffusion models are heavy, plan for 30 to 50 GB. Docker and Docker Compose are required, along with a subdomain (e.g. ia.your-domain.com) and port 443. Prepare the list of models to load according to your uses.
Deploy LocalAI with Docker and HTTPS
Prepare the models folder
Over SSH: mkdir -p /opt/localai/models && cd /opt/localai. This models folder will be mounted in the container and will contain your GGUF files and YAML configurations.
Launch the LocalAI container
Use the official image, for example docker run -d -p 127.0.0.1:8080:8080 -v $PWD/models:/models --name localai localai/localai. Choose the tag matching your hardware (CPU or GPU).
Install a model via the gallery
LocalAI provides a gallery: curl http://127.0.0.1:8080/models/apply -d '{"id":"..."}', or manually drop a GGUF into models/ with its config YAML. Then list via curl http://127.0.0.1:8080/v1/models.
Test OpenAI compatibility
Validate with a chat call: curl http://127.0.0.1:8080/v1/chat/completions -d '{"model":"...","messages":[{"role":"user","content":"test"}]}'. Also test /v1/embeddings if you plan to do RAG.
Configure the reverse proxy and SSL
Route ia.your-domain.com to localhost:8080 via Caddy or Nginx, with Let's Encrypt. Add token protection because LocalAI does not enforce a key by default on a bare deployment.
Migrate your applications
In your OpenAI clients, change the base URL to https://ia.your-domain.com/v1 and the key to your token. Chat, embeddings and image calls switch to your infrastructure.
Only enable the backends you need. Loading a chat model, an embedding model and Stable Diffusion simultaneously on a CPU VPS saturates the RAM and collapses performance. Set SINGLE_ACTIVE_BACKEND=true to keep only one backend resident at a time, and reserve image generation for a GPU VPS if it becomes a regular use.
