Why self-host AnythingLLM on a VPS
AnythingLLM is first and foremost a complete RAG engine: document ingestion, chunking, vectorization, semantic search and generation, all within a multi-user interface. The heart of the matter is the data: financial reports, contracts, product documentation, support knowledge base. On a VPS, the document embedder, the vector database (LanceDB built in by default) and the conversations stay in your volumes. You control who accesses which workspace, you choose the LLM (cloud API or local model), and you avoid the ingestion quotas of SaaS offerings. For an agency managing several clients, each workspace becomes a watertight silo on infrastructure you control.
The concrete benefits of a self-hosted AnythingLLM
- Confidential documents indexed locally, never sent to a third-party ingestion service.
- Workspaces partitioned by client or team, with role management.
- Vector database of your choice: embedded LanceDB, or external Chroma/Qdrant on the same VPS.
- Connection to any LLM provider, including a local Ollama for zero-cloud.
- No limit on the number of documents or pages ingested.
- Simple backup: the entire state fits in a single
storagevolume to archive.
Hardware and software requirements
The AnythingLLM container is reasonable, but embedding large corpora consumes CPU and RAM. Aim for 2 vCPU / 2 GB of RAM to start, and 4 vCPU / 8 GB if you index thousands of documents or use a local embedding model. Plan for 15 to 20 GB of disk, because vectors and the document cache grow quickly. Docker and Docker Compose are required, along with a subdomain (e.g. chat.your-domain.com) and port 443 open. If you stick with cloud APIs for inference, keep your keys handy; for fully local, plan for extra RAM for the model.
Deploy AnythingLLM with Docker and HTTPS
Initialize the environment
Over SSH, create mkdir -p /opt/anythingllm/storage && cd /opt/anythingllm. Adjust the permissions of the storage folder that will host the vector database and the documents: chmod -R 777 storage (the container runs with a dedicated UID).
Write the docker-compose and the .env
Use the mintplexlabs/anythingllm image, map the internal port 3001, mount ./storage:/app/server/storage, and point the STORAGE_DIR variable. In a .env file, define JWT_SECRET and the chosen LLM provider (LLM_PROVIDER, associated keys).
Start and create the admin account
Run docker compose up -d. On first local access on port 3001, AnythingLLM guides you to create the administrator account and configure the embedding model and the LLM.
Set up the reverse proxy
With Nginx Proxy Manager or Caddy, route chat.your-domain.com to localhost:3001 and enable Let's Encrypt. Remember to increase client_max_body_size on the proxy side to allow uploading large documents.
Create a first workspace
In the interface, create a workspace, drag in a few PDFs, run the embedding then test a question. Check that the answers properly cite the sources extracted from your files.
Enable multi-user mode
In the settings, switch to multi-user mode, invite your collaborators and assign roles. Restrict each user to only the workspaces that concern them.
For large corpora, don't stick with embedded LanceDB: deploy Qdrant in a neighboring container and point AnythingLLM to it via the internal Docker network. Qdrant handles millions of vectors better, offers metadata filtering and remains independently queryable, which makes debugging your semantic searches easier.
