How much RAM does LiteLLM need?

LiteLLM itself needs around 256 MB. PostgreSQL (for usage tracking and analytics) adds another 256–512 MB. A 1 GB VPS is the minimum for a comfortable single-user setup; 2 GB is recommended for team deployments with concurrent requests.

Can LiteLLM work with my Ollama instance?

Yes. Add Ollama as a provider in the LiteLLM dashboard — set the base URL to `http://host.docker.internal:11434` (Docker adds this host automatically). Any app using the OpenAI SDK can then call your Ollama models through LiteLLM without changing a single import.

How do I limit spending per team?

Create virtual API keys in the LiteLLM dashboard and set `max_budget` in USD. When a key's cumulative spend reaches the cap, LiteLLM returns a 429. You can also set `tpm_limit` (tokens per minute) and `rpm_limit` (requests per minute) per key for rate-based control.

Is there a built-in dashboard?

Yes. LiteLLM ships a web UI at `/ui` on port 4000. It shows real-time spend by key and by model, request logs, latency breakdowns, and lets you manage models, virtual keys and team budgets — no separate deployment needed.

Does LiteLLM work air-gapped with local models only?

Yes. Configure only Ollama as a provider and remove any external API keys — all inference stays on your VPS. LiteLLM adds routing, load balancing and the dashboard on top of your local Ollama without any outbound connection requirement.

Build, host and run AI solutions.

LiteLLM

Self-hosted LLM API gateway — route between Ollama, Anthropic and 100+ providers from a single OpenAI-compatible endpoint.

1 GB (2 GB recommended) RAM 2 vCPU Port 4000 Available

Source code

Tech stack

DockerPythonPostgreSQL 16

Minimum RAM1 GB (2 GB recommended)

Minimum CPU2 vCPU

Default port4000

Compatible OSubuntu-24.04

LiteLLM is an open-source LLM proxy that presents a unified OpenAI-compatible API for over 100 LLM providers. Your applications keep a single endpoint — and LiteLLM routes each request to Ollama on the same VPS, to Anthropic Claude, to Azure OpenAI, or to any other backend you configure — without a single line of application code changing.

Deployed on a ServOrbit VPS, LiteLLM gives your development team a private AI gateway with a built-in dashboard for usage analytics, per-key spend limits and rate limiting. It runs alongside Ollama for fully air-gapped inference, or in front of commercial APIs to centralize cost tracking across all your projects.

Key features

OpenAI-compatible REST API — drop-in replacement, works with every SDK that speaks OpenAI

100+ LLM providers: Ollama, Anthropic, OpenAI, Azure, Mistral, Cohere, Bedrock and more

Per-key rate limiting and budget caps — block runaway costs in multi-team or multi-project setups

Real-time spend analytics and cost dashboard with per-model and per-key breakdowns

Load balancing and automatic failover across provider endpoints

Apache 2.0 — fully auditable, no telemetry, air-gap capable with Ollama

When to use this solution?

1

Multi-model AI backend

Route cost-sensitive tasks to a local Ollama instance and complex reasoning requests to Anthropic Claude — all from the same OpenAI SDK call in your application. Change the routing in the LiteLLM config, never in app code.

2

Team AI budget control

Issue virtual API keys to each team or project with a monthly spend cap. When a key hits its budget, LiteLLM returns a 429 — no surprise cloud bill at the end of the month.

3

LLM provider migration

Switch your entire application from one provider to another by updating a config file. OpenAI-compatible surface means zero application changes — migrate from GPT-4 to Llama 3 on Ollama without touching a single import.

Deploy LiteLLM on your VPS

Guide optimized for ServOrbit Cloud VPS.

01

Provision a VPS

Order a ServOrbit VPS with at least 1 GB RAM running Ubuntu 22.04. LiteLLM itself needs around 256 MB; PostgreSQL takes most of the remaining budget. Port 4000 must be reachable, or plan a reverse proxy on 80/443.

02

Generate secrets

SSH into the VPS and run openssl rand -hex 32 to generate LITELLM_MASTER_KEY — this is your admin API key that unlocks the dashboard and all administrative endpoints. Also set strong DB credentials. Keep the master key safe.

03

Deploy with Docker Compose

Create /opt/litellm/.env with DB_NAME, DB_USER, DB_PASSWORD, LITELLM_MASTER_KEY, UI_USERNAME and UI_PASSWORD. Then create /opt/litellm/docker-compose.yml with two services: LiteLLM (ghcr.io/berriai/litellm:main-latest, port 4000) and PostgreSQL 16. Run docker compose up -d. Allow 30 seconds for first boot.

04

Open the dashboard and add your first model

Navigate to http://:4000/ui and log in with your UI_USERNAME and UI_PASSWORD. Go to Models, click Add — enter a model name like ollama/llama3.1 with base URL http://host.docker.internal:11434 to route to a local Ollama, or paste an Anthropic API key for anthropic/claude-3-5-haiku-20241022. Test with curl -H "Authorization: Bearer " http://:4000/v1/models.

05

Issue team API keys

In the dashboard under API Keys, create per-team virtual keys with optional max_budget (in USD), tpm_limit and rpm_limit. Share these keys with your developers — they point their OpenAI SDK at your LiteLLM gateway and the spend is tracked per key.

Frequently asked questions

LiteLLM is an open-source LLM proxy (Apache 2.0) that exposes a unified OpenAI-compatible API for over 100 LLM providers — Ollama, Anthropic Claude, OpenAI GPT-4, Azure OpenAI, Mistral, Cohere and more. It sits between your applications and any LLM backend, handling routing, rate limiting and cost tracking.

Complementary solutions

Artificial Intelligence

Activate LiteLLM on your infrastructure.

Dedicated Cloud VPS — IPv4 included, European datacenter, support included. Your data never leaves your server.

Recommended configuration: 1 GB (2 GB recommended) RAM · 2 vCPU