Why self-host PrivateGPT on a VPS
PrivateGPT is designed around RAG (Retrieval-Augmented Generation): it ingests your documents, splits them into fragments, computes embeddings stored in a vector database (Qdrant by default), then answers questions relying solely on this corpus. Its central argument is confidentiality: in local mode, the LLM and the embeddings run on the machine, no request goes out. On a dedicated VPS, you get a secure enterprise documentary assistant, isolated from everyone's workstation, with a centralized knowledge base that the whole team queries, without ever exposing contracts, HR files or legal documents to a cloud service.
Concrete benefits of self-hosting
- Total confidentiality: documents and embeddings never leave the VPS
- Centralized Qdrant vector database, queryable by the whole team in one place
- No API cost if you use a local LLM via Ollama or llama.cpp
- Ingestion of PDF, DOCX, emails and web pages into a single private corpus
- OpenAI-compatible API to connect PrivateGPT into your internal tools
- Easier compliance for sensitive data (GDPR, professional secrecy)
Hardware and software requirements
PrivateGPT in fully local mode is more demanding than other tools because it runs both the embedding model and the generation LLM. Count on at least 8 GB of RAM (16 GB recommended) and 4 vCPU for a quantized 7B model, plus 20 GB of disk for the models, the Qdrant database and the ingested documents. If you offload the LLM to an Ollama on the same VPS, size the RAM accordingly. On the software side: Docker and Docker Compose, Python 3.11 if you install outside a container, and a domain pointing to the VPS to expose the Gradio interface and the API.
Deployment of PrivateGPT in local mode
Prepare the VPS
Install Docker and clone the repository: git clone https://github.com/zylon-ai/private-gpt && cd private-gpt. Create a persistent volume for the ingested data and the Qdrant database so as to survive redeployments.
Choose the execution profile
PrivateGPT works by profiles. For a 100% local deployment with Ollama, start Ollama then pull the models: ollama pull mistral for generation and ollama pull nomic-embed-text for the embeddings.
Configure settings and launch
Enable the ollama profile in settings.yaml then start with PGPT_PROFILES=ollama make run or via the provided compose. Check in the logs that the connection to Ollama and to Qdrant is established.
Ingest your documents
Drop your files in the ingestion folder and run make ingest /chemin/vers/docs. PrivateGPT splits, vectorizes and indexes the corpus in Qdrant. Follow the progress in the logs until the indexing is complete.
Set up the reverse proxy and SSL
Expose the Gradio interface (port 8001) behind Caddy or Nginx with an automatic Let's Encrypt certificate. Add basic HTTP authentication, because the UI does not offer it natively.
Lock down access
Close port 8001 at the firewall to leave only the reverse proxy reachable. Your users access https://docs.yourdomain.com and query the corpus in natural language.
The default profile downloads heavy models. For a CPU VPS, switch the LLM and the embeddings to Ollama (mistral + nomic-embed-text) rather than the llama.cpp bindings compiled locally: you avoid the long compilations, pool the memory with other containers and gain stability during restarts.
