What LLMs does AnythingLLM support?

Ollama (local models), OpenAI, Anthropic, Azure OpenAI, Mistral, Google Gemini, LiteLLM, Cohere, TogetherAI, Groq, Llama.cpp, and more. You can switch providers per workspace without losing your documents.

What is the minimum RAM?

AnythingLLM itself needs ~512 MB RAM. For document embeddings and inference, you need an LLM: if you use Ollama locally with a 7B model, plan for 8 GB total. If you use an external API (OpenAI, Anthropic), 2 GB RAM is sufficient.

Can it handle large document collections?

Yes. AnythingLLM uses LanceDB for its built-in vector store, which scales to millions of vectors on a single VPS. For very large collections you can swap the vector store to Qdrant, Pinecone, or Weaviate in Settings → Vector DB.

The self-hosted version is MIT-licensed and completely free. Mintplex Labs also offers a cloud-hosted plan and a desktop app, but everything you need for a private deployment is in the open-source Docker image.

Build, host and run AI solutions.

AnythingLLM

Private AI workspace with RAG and agents — your documents, your models, your server.

512 MB (2 GB+ with local Ollama) RAM 1 vCPU Port 3001 Available

Source code

Tech stack

DockerNode.jsPythonSQLiteLanceDB

Minimum RAM512 MB (2 GB+ with local Ollama)

Minimum CPU1 vCPU

Default port3001

Compatible OSChoice of Linux distributions

AnythingLLM is an all-in-one open-source AI application that turns your documents into a private, queryable knowledge base. It ships as a single Docker image that runs on any 2 GB VPS and connects to whichever LLM you already use — Ollama (local), OpenAI, Anthropic, Mistral, LiteLLM, and more.

The built-in RAG engine ingests PDFs, Word docs, plain text, code repositories, YouTube transcripts, Notion pages, and Confluence spaces, then chunks, embeds, and indexes them into an integrated LanceDB vector store. Each workspace is isolated: different documents, different LLM, different system prompt. Ask a question and AnythingLLM retrieves the relevant chunks, passes them as context to your model, and returns a cited answer.

AnythingLLM also ships a no-code AI agent builder: string together web search, code execution, document retrieval, and external API calls without writing a line of code. User management with roles (admin, manager, default) makes it suitable for team deployments. MIT-licensed and 62k+ GitHub stars as of July 2026.

Key features

Built-in RAG: ingest PDFs, Word docs, TXT, code repos, YouTube, Confluence, Notion, and more

Connect any LLM: Ollama, OpenAI, Anthropic, Mistral, Azure OpenAI, LiteLLM, Groq, and local models

No-code AI agent builder with web search, code execution, and API call tools

Multi-workspace isolation: separate documents, LLM, and system prompt per workspace

Multi-user with roles: admin, manager, default user — team-ready out of the box

REST API: embed AnythingLLM into your own products via a documented HTTP API

Pluggable vector store: built-in LanceDB (zero setup) or connect Qdrant, Pinecone, Chroma, Weaviate

Zero telemetry, MIT licence, fully air-gap capable

When to use this solution?

1

Internal knowledge base

Upload your SOPs, contracts, and technical docs. Your team asks questions in natural language and gets accurate, cited answers — without sending any data to OpenAI.

2

Developer assistant

Ingest your codebase, architecture docs, and API references. Use AnythingLLM as a private coding assistant that knows your stack and answers in context.

3

Customer-facing AI

Expose AnythingLLM's REST API to power an AI chat widget on your product, trained exclusively on your own documentation.

Deploy AnythingLLM on your VPS

Guide optimized for ServOrbit Cloud VPS.

01

Start the AnythingLLM container

Connect via SSH and run: docker run -d --restart=always -p 3001:3001 --cap-add SYS_ADMIN -v anythingllm-storage:/app/server/storage -e STORAGE_DIR=/app/server/storage --name anythingllm mintplexlabs/anythingllm:latest. The image (~700 MB) downloads once and starts in a few seconds. The named volume anythingllm-storage persists all documents, settings, and the SQLite database.

02

Create your admin account

Open http://your-vps-ip:3001 in your browser. AnythingLLM detects a fresh install and walks you through creating the first admin account. This also sets the JWT secret used to sign all user sessions.

03

Connect an LLM provider

Go to Settings → LLM Provider. Choose Ollama (if running on the same VPS), OpenAI (enter your API key), or any supported provider. AnythingLLM tests the connection before saving. You can also point it at LiteLLM to switch providers without touching your workspace settings.

04

Create a workspace and upload documents

Click New Workspace, give it a name (e.g. "Product Docs"), then drag-and-drop PDFs or paste a URL. AnythingLLM chunks the content, generates embeddings, and stores them in LanceDB. Ask a question — the system retrieves the relevant chunks and passes them to your LLM.

05

Optional: add HTTPS with Caddy

Create a Caddyfile: ai.yourdomain.com { reverse_proxy localhost:3001 }. Run caddy run. Caddy automatically issues a Let's Encrypt certificate. AnythingLLM now serves over HTTPS on your custom domain.

Frequently asked questions

AnythingLLM is a self-hosted AI workspace that combines document RAG, multi-model chat, and AI agents in a single Docker container. You upload your documents, connect any LLM provider, and interact with your data in natural language — everything stays on your server.