[{"data":1,"prerenderedAt":118},["ShallowReactive",2],{"seo-verification":3,"marketplace-app-en-local-ai":6},{"google":4,"bing":5},"EycwPY2XMyTkVzas3n1ygeNJFGAH513qrMjfDljzsMQ","",{"slug":7,"slugs":8,"categorySlugs":9,"name":13,"description":14,"phase":15,"unavailableReason":16,"docsUrl":17,"logo":18,"github":19,"tagline":20,"longDescription":21,"features":22,"useCases":30,"steps":40,"faq":62,"specs":81,"compatibleOs":89,"relatedApps":90,"relatedPosts":111,"category":115},"local-ai",{"fr":7,"en":7,"ar":7},{"fr":10,"en":11,"ar":12},"ia","artificial-intelligence","الذكاء-الاصطناعي","LocalAI","Run open-source LLMs on CPU — no GPU required. Drop-in OpenAI-compatible API, 200+ models (Llama, Mistral, Phi), fully self-hosted on your VPS.",1,null,"https:\u002F\u002Fservorbit.com\u002Fblog\u002Fhow-to-host-localai-on-a-vps","https:\u002F\u002Fraw.githubusercontent.com\u002Fmudler\u002FLocalAI\u002Fmaster\u002Fcore\u002Fhttp\u002Fstatic\u002Flogo.png","https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI","Run open-source LLMs on CPU — no GPU required. OpenAI-compatible API, 200+ models, self-hosted in one command.","LocalAI (MIT, ~47k GitHub stars) is a free, open-source alternative to the OpenAI API. It runs on consumer-grade hardware without a GPU — any VPS with 2 GB RAM can serve Llama 3, Mistral, Phi-3, Qwen and 200+ other models via an OpenAI-compatible REST API. Your existing code that calls `openai.ChatCompletion.create()` works without modification: change the base URL to your VPS and all API calls are routed locally, with zero per-token cost and complete data privacy.\n\nDeployed on a ServOrbit VPS, LocalAI becomes your private LLM backend: an endpoint your team's applications, RAG pipelines and agent frameworks call instead of OpenAI — with no cloud dependency, no usage limits, and no data leaving your infrastructure. LocalAI supports text generation, function calling, image generation (Stable Diffusion), speech-to-text (Whisper), and text embeddings — all from a single container.",[23,24,25,26,27,28,29],"OpenAI-compatible REST API — drop-in replacement for OpenAI, works with every LangChain\u002FLlamaIndex\u002FLiteLLM SDK unchanged.","CPU-only inference — runs on any VPS without a GPU using llama.cpp; optimised SIMD paths for Intel and ARM.","200+ models supported — Llama 3, Mistral, Phi-3, Qwen 2.5, DeepSeek, Gemma 2, Falcon, and any GGUF-format model.","Multi-modal: text generation, function\u002Ftool calling, embeddings, image generation (Stable Diffusion), and speech-to-text (Whisper).","Pull models by name — `curl http:\u002F\u002Flocalhost:8080\u002Fmodels\u002Fapply -d '{\"id\":\"llama-3.2-3b-instruct:q4_0\"}'` downloads and activates any supported model.","MIT licence — fully auditable, no telemetry, air-gap capable; v4.6.2 (released 2026-07-06) adds distributed cluster mode and prefix-cache-aware routing.","Composable with LiteLLM and Open WebUI — LocalAI serves the inference layer; LiteLLM adds multi-provider routing; Open WebUI adds the chat UI.",[31,34,37],{"title":32,"body":33},"Private LLM backend for your applications","Replace your OpenAI API calls with a LocalAI endpoint on your VPS. Your SaaS, internal tool, or RAG pipeline sends requests to `https:\u002F\u002Fyour-domain.com\u002Fv1\u002Fchat\u002Fcompletions` — identical JSON, identical SDK, zero cost per token. Reaching the endpoint from another machine requires a domain attached to the VPS, since LocalAI only listens on the loopback; code running on the VPS itself calls `http:\u002F\u002F127.0.0.1:8080\u002Fv1\u002Fchat\u002Fcompletions`. Ideal for high-volume use cases where per-token API pricing becomes a bottleneck.",{"title":35,"body":36},"Air-gapped inference for sensitive data","For legal, medical or financial applications where data cannot leave your infrastructure, LocalAI runs entirely offline once the model is downloaded. No requests ever reach an external API. The GGUF model files are stored in a Docker volume on your own disk.",{"title":38,"body":39},"Embeddings and semantic search without an API key","Run embedding models (nomic-embed-text, mxbai-embed-large) locally alongside Qdrant on the same VPS to build a complete semantic search or RAG pipeline. No OpenAI embedding API cost, no data leaving your server, no rate limits.",[41,44,47,50,53,56,59],{"title":42,"body":43},"Create the VPS","Order a ServOrbit VPS with at least 2 GB of RAM; the system installed is Ubuntu 24.04. For comfortable multi-user use, or for models from 7 billion parameters upwards, 4 GB is recommended. Core count matters for speed: 4 vCPU noticeably reduce generation latency on models of 3 to 7 billion parameters.",{"title":45,"body":46},"Deploy LocalAI","Installing from your client area creates the `localai\u002Flocalai:latest` container — an image of about 900 MB — mounts the `local-ai` volume on `\u002Fbuild\u002Fmodels`, and publishes the OpenAI-compatible API server on the loopback interface, at `127.0.0.1:8080`. No model ships with it: the first request naming a model triggers its download into the persistent volume.",{"title":48,"body":49},"Connect for the first time","LocalAI opens without any credentials at all: your first move is to go through the model gallery and download one, since the installation provides none. The OpenAI-compatible API answers at the same address and is protected by no key — as long as the service listens only on the loopback interface, only users with SSH access to the VPS can reach it.",{"title":51,"body":52},"Pull a model","A single API call is enough: `curl http:\u002F\u002F127.0.0.1:8080\u002Fmodels\u002Fapply -H 'Content-Type: application\u002Fjson' -d '{\"id\":\"llama-3.2-3b-instruct:q4_0\"}'`. LocalAI fetches the GGUF file and registers the model. The Q4 quantisation of a 3 billion parameter model fits in 2 GB of RAM; a 7 billion one calls for about 4.",{"title":54,"body":55},"Make your first API call","Test the API from the VPS: `curl http:\u002F\u002F127.0.0.1:8080\u002Fv1\u002Fchat\u002Fcompletions -H 'Content-Type: application\u002Fjson' -d '{\"model\":\"llama-3.2-3b-instruct:q4_0\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello!\"}]}'`. The response has exactly the shape of OpenAI's: changing the base URL in your existing code is all it takes to make it work.",{"title":57,"body":58},"Point your apps at LocalAI","In any OpenAI SDK, set `base_url='https:\u002F\u002F\u003Cyour-domain>\u002Fv1'` and `api_key='not-needed'`; that public address assumes a domain attached to the VPS. From the VPS itself, use `base_url='http:\u002F\u002F127.0.0.1:8080\u002Fv1'`. Without a domain, open an SSH tunnel — `ssh -L 8080:127.0.0.1:8080 root@\u003Cyour-vps-ip>` — and point at `http:\u002F\u002Flocalhost:8080\u002Fv1`. The port may be reassigned at install time: use the one shown on the app's card in your client area, 8080 being only the catalogue value. In LangChain that gives `ChatOpenAI(base_url=..., api_key='x')`; in LiteLLM, prefix the model with `localai\u002F`; in Open WebUI, declare an OpenAI connection with that same base URL. Your existing code needs no other change.",{"title":60,"body":61},"Protect the endpoint","LocalAI is open by default: no key is required. The application knows how to demand one through the `LOCALAI_API_KEY` environment variable, but the catalogue installation does not set one — so treat the endpoint as unauthenticated. As soon as a domain is attached, ServOrbit installs the nginx vhost and the TLS certificate; encryption protects the transport, it identifies nobody. As long as no key is in place, stay without a domain and work through an SSH tunnel, or publish an authenticating gateway instead — LiteLLM for instance — and keep LocalAI behind it.",[63,66,69,72,75,78],{"q":64,"a":65},"What is LocalAI?","LocalAI is a free, open-source (MIT) server that runs large language models locally on any CPU. It exposes an OpenAI-compatible REST API, so your existing code works without modification — just change the API base URL to your server.",{"q":67,"a":68},"Do I need a GPU to run LocalAI?","No. LocalAI is built on llama.cpp, which runs efficiently on CPU using SIMD instructions (AVX2\u002FAVX-512 on Intel, NEON on ARM). A VPS with 2–4 GB RAM and 2–4 vCPU is enough for 3B–7B models. GPU acceleration is optional and supported if your server has a CUDA-compatible GPU.",{"q":70,"a":71},"Which models can I run?","LocalAI supports any model in GGUF format, which covers virtually all open-source LLMs: Llama 3.x, Mistral 0.7–7B, Phi-3\u002F3.5\u002F4 Mini, Qwen 2.5, DeepSeek-R1, Gemma 2, Falcon, and more. Pull a model with one API call — LocalAI downloads it from Hugging Face automatically.",{"q":73,"a":74},"How does LocalAI compare to Ollama?","Both run LLMs locally on CPU. LocalAI's advantage is its broader compatibility: it supports more model backends (not just llama.cpp), image generation, speech-to-text (Whisper), and embeddings from a single API. Ollama is simpler to set up and has a larger default model library. Both expose an OpenAI-compatible API.",{"q":76,"a":77},"Is my data private?","Yes. All inference happens on your VPS — no request is ever sent to an external API. The models are stored in a Docker volume on your own disk. LocalAI has no telemetry by default (MIT licence, fully auditable source code).",{"q":79,"a":80},"What are the RAM requirements?","A 3B model at Q4 quantisation needs roughly 2 GB RAM. A 7B model needs ~4 GB. A 13B model needs ~8 GB. For the smallest ServOrbit VPS plans (2 GB), the Phi-3 Mini (3.8B at Q4) runs well and delivers strong reasoning performance relative to its size.",{"ram":82,"cpu":83,"stack":84,"port":88},"2 GB (4 GB recommended for 7B+ models)","2 vCPU (4 vCPU recommended)",[85,86,87],"Docker","Go","llama.cpp","8080",[],[91,99,105],{"name":92,"slug":93,"categorySlug":11,"categoryName":94,"categoryColor":95,"logo":96,"tagline":97,"description":98},"Ollama","ollama","Artificial Intelligence","text-purple-400 bg-purple-500\u002F10","https:\u002F\u002Fcdn.simpleicons.org\u002Follama","Run open-source LLMs on your VPS — OpenAI-compatible API, zero per-token cost.","Run open-source LLMs locally via a dead-simple API. Pull Llama 3, Mistral, Qwen or DeepSeek in one command — OpenAI-compatible, zero per-token cost.",{"name":100,"slug":101,"categorySlug":11,"categoryName":94,"categoryColor":95,"logo":102,"tagline":103,"description":104},"LiteLLM","litellm","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F132372032","Self-hosted LLM API gateway — route between Ollama, Anthropic and 100+ providers from a single OpenAI-compatible endpoint.","Self-hosted OpenAI-compatible API gateway for 100+ LLMs — route between Ollama, Anthropic, Azure and more from a single endpoint, with per-key rate limits and spend tracking.",{"name":106,"slug":107,"categorySlug":11,"categoryName":94,"categoryColor":95,"logo":108,"tagline":109,"description":110},"Open WebUI","open-webui","https:\u002F\u002Fcdn.jsdelivr.net\u002Fgh\u002Fselfhst\u002Ficons\u002Fsvg\u002Fopen-webui.svg","Web interface for your LLMs — Ollama, OpenAI, Mistral — hosted on your own server.","Web interface to interact with your local or remote LLMs. Your data stays on your infrastructure — no third party involved.",[112,113,114],"self-host-litellm-vps","heberger-ollama-vps","deployer-dify-vps",{"key":10,"slug":11,"name":94,"objective":116,"icon":117,"color":95},"Build, host and run AI solutions.","ai",1787661758080]