Artificial Intelligence

PrivateGPT

100% local AI document assistant — RAG on your files, no cloud, no data leak.

Official documentation

About PrivateGPT

PrivateGPT is an open-source RAG (Retrieval-Augmented Generation) platform that lets you chat with your own documents — PDFs, DOCX, spreadsheets and emails — using a local large language model. Unlike cloud-based AI assistants, PrivateGPT never sends your data outside the server: ingestion, vectorization, storage and inference all happen on your VPS. Powered by an embedded vector store and connected to any OpenAI-compatible inference server (Ollama, llama.cpp, vLLM), it gives you a full RAG pipeline under your control.

Deployed on a ServOrbit VPS alongside Ollama, PrivateGPT becomes a self-contained private AI workspace: upload your contracts, technical documentation or meeting notes, and query them in natural language. Each answer cites the source passage and file name, so you know exactly where it came from. The OpenAI-compatible API at port 8080 lets you integrate the same document intelligence into your internal tools or build custom agents on top of the same indexed corpus.

What gets installed

Ready when the VPS starts

Compatible OS
Choice of Linux distributions
Application
PrivateGPT
Tech stack
DockerPythonOllamaVector store
Minimum RAM
8 GB
Minimum CPU
2 vCPU
Default port
8080
Access
root over SSH

Key features

  • Full offline RAG pipeline — ingest PDFs, DOCX, TXT and CSV; query with your local LLM, zero cloud calls
  • Source citations on every answer — each response includes the passage and file name that supported it
  • OpenAI-compatible API at port 8080 — drop-in replacement for the OpenAI client in your existing tools
  • Connects to any OpenAI-compatible server — Ollama, llama.cpp, vLLM, or the real OpenAI API
  • Embedded vector store — semantic chunking and embedding stored on-disk, persistent across restarts
  • Apache-2.0 licence — fully auditable, no telemetry, air-gap capable, 57 k+ GitHub stars

When to use this solution?

Legal and HR document intelligence

Upload contracts, employment agreements and HR files to PrivateGPT. Ask 'What are the termination clauses in contract X?' and get a cited answer without any data leaving the VPS — fully compliant with GDPR Article 28.

Internal knowledge base

Index your team's technical documentation, runbooks and meeting notes. Every team member gets a natural-language interface to the corpus, with answers pointing to the exact source passage so they can verify and go deeper.

Private code and security review

Feed proprietary source code or security audit reports into PrivateGPT. Ask questions about architecture, find undocumented functions, or cross-reference CVE descriptions against your own codebase — without a single line of source code leaving the server.

Guide

Deploy PrivateGPT on your VPS

  1. Order a ServOrbit VPS

    Allocate at least 8 GB of RAM: Ollama needs 4–6 GB for a 7B model (Q4_K_M), PrivateGPT itself takes around 500 MB, and you need headroom for document ingestion. A 16 GB VPS is the comfortable choice. The machine ships with Ubuntu 24.04 and Docker pre-installed.

  2. Deploy from the Marketplace

    In your ServOrbit client area: Marketplace → Artificial Intelligence → PrivateGPT → Deploy. The stack starts two containers: PrivateGPT (web UI + API on port 8080, bound to 127.0.0.1) and an Ollama sidecar (internal only, not reachable from the internet). Both share a private Docker network — only port 8080 is exposed on the loopback.

  3. Pull your first model

    Once the stack is up, pull a language model into Ollama via SSH: docker exec ollama ollama pull qwen2.5:7b. For a lightweight start on 8 GB RAM, qwen2.5:3b (2.3 GB) is a good entry point. For richer answers on long documents, mistral:7b or llama3.1:8b are recommended. For embeddings, nomic-embed-text improves retrieval quality.

  4. Open the web UI

    PrivateGPT's interface is at http://127.0.0.1:8080/ui (loopback only). Open an SSH tunnel from your workstation: ssh -L 8080:127.0.0.1:<port> root@<vps-ip> (replace <port> with the port shown in your client area), then go to http://localhost:8080/ui in your browser.

  5. Ingest your documents

    Use the Upload button in the UI or the /v1/ingest/file API endpoint to upload your documents. PrivateGPT splits each file into chunks, computes embeddings via Ollama, and stores them in the local vector store. Ingestion of a 50-page PDF typically takes 30–90 seconds on CPU.

  6. Query your corpus

    Type your question in the chat. PrivateGPT retrieves the most relevant chunks from the vector store, passes them as context to the LLM, and returns an answer with source passages cited below. The /v1/chat/completions endpoint is OpenAI-compatible for API clients.

Frequently asked questions

Questions about PrivateGPT

← Back to Artificial IntelligenceAll applications
No. CPU-only inference works out of the box via Ollama. A modern server CPU handles a 7B Q4_K_M model at 2–5 tokens/sec — usable for document Q&A where accuracy matters more than speed. For faster inference on larger models, a GPU-equipped VPS accelerates throughput 10–30×.

Embed the deploy button

Maintaining a project that uses PrivateGPT? This button lets your readers deploy it on a VPS in one click, without reading Docker documentation.

Deploy PrivateGPT on ServOrbit
Markdown
[![Deploy PrivateGPT on ServOrbit](https://servorbit.com/brand/deploy/button.svg)](https://servorbit.com/vps-cloud?template=private-gpt&utm_source=deploy-badge&utm_medium=referral&utm_campaign=private-gpt)
HTML
<a href="https://servorbit.com/vps-cloud?template=private-gpt&utm_source=deploy-badge&utm_medium=referral&utm_campaign=private-gpt"><img src="https://servorbit.com/brand/deploy/button.svg" alt="Deploy PrivateGPT on ServOrbit" height="40"></a>

The button points to a VPS order with the template preselected. The image is served from servorbit.com — nothing to host on your side.

Complementary solutions

Artificial Intelligence

AnythingLLM

Open-source AI workspace with built-in RAG, AI agents, and multi-model chat. Connect your own documents, use any LLM — hosted privately on your VPS.

Artificial Intelligence

Open WebUI

Web interface to interact with your local or remote LLMs. Your data stays on your infrastructure — no third party involved.

Artificial Intelligence

Dify

Build LLM applications visually: chatbots, RAG agents, pipelines — without writing any backend code.

Need help?

Browse our help center and FAQ, or reach our team — callback, WhatsApp or email.

Message us on WhatsAppopens in a new tab
or write to us