What LLMs does AnythingLLM support?

Ollama (local models), OpenAI, Anthropic, Azure OpenAI, Mistral, Google Gemini, LiteLLM, Cohere, TogetherAI, Groq, Llama.cpp, and more. You can switch providers per workspace without losing your documents.

What is the minimum RAM?

AnythingLLM itself needs ~512 MB RAM. For document embeddings and inference, you need an LLM: if you use Ollama locally with a 7B model, plan for 8 GB total. If you use an external API (OpenAI, Anthropic), 2 GB RAM is sufficient.

Can it handle large document collections?

Yes. AnythingLLM uses LanceDB for its built-in vector store, which scales to millions of vectors on a single VPS. For very large collections you can swap the vector store to Qdrant, Pinecone, or Weaviate in Settings → Vector DB.

The self-hosted version is MIT-licensed and completely free. Mintplex Labs also offers a cloud-hosted plan and a desktop app, but everything you need for a private deployment is in the open-source Docker image.

Build, host and run AI solutions.

AnythingLLM

Private AI workspace with RAG and agents — your documents, your models, your server.

512 MB (2 GB+ with local Ollama) RAM 1 vCPU Port 3001 Available

Source code

Tech stack

DockerNode.jsPythonSQLiteLanceDB

Minimum RAM512 MB (2 GB+ with local Ollama)

Minimum CPU1 vCPU

Default port3001

Compatible OSChoice of Linux distributions

AnythingLLM is an all-in-one open-source AI application that turns your documents into a private, queryable knowledge base. It ships as a single Docker image that runs on any 2 GB VPS and connects to whichever LLM you already use — Ollama (local), OpenAI, Anthropic, Mistral, LiteLLM, and more.

The built-in RAG engine ingests PDFs, Word docs, plain text, code repositories, YouTube transcripts, Notion pages, and Confluence spaces, then chunks, embeds, and indexes them into an integrated LanceDB vector store. Each workspace is isolated: different documents, different LLM, different system prompt. Ask a question and AnythingLLM retrieves the relevant chunks, passes them as context to your model, and returns a cited answer.

AnythingLLM also ships a no-code AI agent builder: string together web search, code execution, document retrieval, and external API calls without writing a line of code. User management with roles (admin, manager, default) makes it suitable for team deployments. MIT-licensed and 62k+ GitHub stars as of July 2026.

Key features

Built-in RAG: ingest PDFs, Word docs, TXT, code repos, YouTube, Confluence, Notion, and more

Connect any LLM: Ollama, OpenAI, Anthropic, Mistral, Azure OpenAI, LiteLLM, Groq, and local models

No-code AI agent builder with web search, code execution, and API call tools

Multi-workspace isolation: separate documents, LLM, and system prompt per workspace

Multi-user with roles: admin, manager, default user — team-ready out of the box

REST API: embed AnythingLLM into your own products via a documented HTTP API

Pluggable vector store: built-in LanceDB (zero setup) or connect Qdrant, Pinecone, Chroma, Weaviate

Zero telemetry, MIT licence, fully air-gap capable

When to use this solution?

1

Internal knowledge base

Upload your SOPs, contracts, and technical docs. Your team asks questions in natural language and gets accurate, cited answers — without sending any data to OpenAI.

2

Developer assistant

Ingest your codebase, architecture docs, and API references. Use AnythingLLM as a private coding assistant that knows your stack and answers in context.

3

Customer-facing AI

Expose AnythingLLM's REST API to power an AI chat widget on your product, trained exclusively on your own documentation.

Deploy AnythingLLM on your VPS

Guide optimized for ServOrbit Cloud VPS.

01

Order a ServOrbit VPS

AnythingLLM itself makes do with about 512 MB of RAM. Sizing depends on the model: with an external API (OpenAI, Anthropic, Mistral), 2 GB is more than enough; with Ollama installed on the same VPS and a 7-billion-parameter model, count on 8 GB in total. Plan disk space for your documents and their vectors.

02

Deploy from the Marketplace

In your ServOrbit client area: Marketplace → AI → AnythingLLM → Deploy. The mintplexlabs/anythingllm:latest image (about 700 MB) is pulled once and the container starts within seconds on port 3001. The named volume anythingllm-storage keeps all of your documents, your settings and the SQLite database.

03

Sign in for the first time

There are no credentials to hand over to you: this installation starts in single-user mode with NO password. Open it straight away, finish the setup wizard, then go to Settings and turn on “Password Protect Instance” — until you do, anyone who reaches the address can read your documents and change your settings.

04

Create the administrator account on first access

Open the application in your browser. With a domain attached, the address is https://<your-domain>; without a domain there is no public address: open an SSH tunnel with ssh -L 3001:127.0.0.1:3001 root@<your-vps-ip>, then go to http://localhost:3001 — 3001 is the catalogue port, use the one shown on your app's card in the client area. AnythingLLM detects a fresh install and guides you through creating the first administrator account; that is also the moment when the JWT secret used to sign every user session is set. No domain is required for this step.

05

Connect a model provider

Go to Settings → LLM Provider. Choose Ollama if it runs on the same VPS, OpenAI by entering your API key, or any other supported provider. AnythingLLM tests the connection before saving. By pointing it at LiteLLM, you will be able to change provider later without touching your workspaces.

06

Create a workspace and upload your documents

Click New Workspace, give it a name (“Product documentation”, for instance), then drop in your PDFs or paste a URL. AnythingLLM chunks the content, computes the vectors and stores them in LanceDB. Ask your first question: the system retrieves the relevant passages and passes them on to your model. Then create the accounts for your team and assign their roles.

Frequently asked questions

AnythingLLM is a self-hosted AI workspace that combines document RAG, multi-model chat, and AI agents in a single Docker container. You upload your documents, connect any LLM provider, and interact with your data in natural language — everything stays on your server.

Embed the deploy button

Maintaining a project that uses AnythingLLM? This button lets your readers deploy it on a VPS in one click, without reading Docker documentation.

Markdown

[![Deploy AnythingLLM on ServOrbit](https://servorbit.com/brand/deploy/button.svg)](https://servorbit.com/vps-cloud?template=anything-llm&utm_source=deploy-badge&utm_medium=referral&utm_campaign=anything-llm)

HTML

<a href="https://servorbit.com/vps-cloud?template=anything-llm&utm_source=deploy-badge&utm_medium=referral&utm_campaign=anything-llm"><img src="https://servorbit.com/brand/deploy/button.svg" alt="Deploy AnythingLLM on ServOrbit" height="40"></a>

The button points to a VPS order with the template preselected. The image is served from servorbit.com — nothing to host on your side.

Complementary solutions

Artificial Intelligence

Activate AnythingLLM on your infrastructure.

Dedicated Cloud VPS — IPv4 included, European datacenter, support included. Your data never leaves your server.

Recommended configuration: 512 MB (2 GB+ with local Ollama) RAM · 1 vCPU