Why LLM applications need dedicated observability
Traditional APM tools (Datadog, New Relic) capture HTTP latency and error rates — but they are blind to what happens inside an LLM call. A response that arrives in 800 ms might still be factually wrong, unhelpfully vague, or three times more expensive than yesterday's because a prompt regression slipped through code review. Langfuse solves this by treating each LLM interaction as a structured trace: it records the full prompt (including system message and conversation history), the completion, the model used, the token counts, the latency breakdown per span, and any evaluation scores your team attaches. You can then filter, compare, and reproduce any trace — individually or in aggregate.
What Langfuse gives you out of the box
- Full-stack LLM tracing: prompt, completion, latency, cost and token count — including nested spans for agent chains (LangChain, LlamaIndex, Dify).
- Prompt management hub: version-control your prompts, stage variants, A/B test in production, promote the winner without a code deploy.
- Evaluation framework: run LLM-as-a-judge, human annotation queues, or custom scoring functions on any trace or dataset.
- Cost analytics: track token spend by model, endpoint, user and session — switch providers with data, not guesswork.
- Dataset management: capture production traces as golden test sets for offline evaluation and regression detection.
- Native SDK for Python and TypeScript, plus automatic integration with LiteLLM, LangChain, LlamaIndex, Dify, Haystack and VercelAI.
What Langfuse runs on (and why it needs 4 GB RAM)
Langfuse v3 ships as a six-service Docker Compose stack: langfuse (Next.js frontend + API), langfuse-worker (background jobs and evaluations), postgres (application state), clickhouse (trace analytics — columnar storage optimised for high-cardinality time-series), redis (queue and cache), and minio (S3-compatible blob storage for media attachments). ClickHouse is the reason for the 4 GB minimum: its JIT compiler and vector execution engine need headroom. On a quiet VPS the full stack idles around 1.5–2 GB; under moderate production load plan for 4 GB, with 8 GB for sustained high-throughput tracing. All six services start from a single docker compose up -d and are managed by AWX in the ServOrbit one-click flow.
Deploy Langfuse on your VPS in six steps
Order a VPS Power (8 GB RAM)
Langfuse needs at least 4 GB of RAM: in the catalogue the first plan that clears that bar is the VPS Power (4 vCPU, 8 GB), installed on Ubuntu 24.04 — enough to absorb persistent production traffic too. Pick a domain or subdomain you control — Langfuse's NextAuth session cookies require a proper HTTPS domain.
One-click install from the marketplace
Open your ServOrbit control panel, go to Marketplace → Artificial Intelligence → Langfuse, and click Deploy. Enter your domain when prompted. Docker Compose pulls all six images and starts them; the web UI is ready on port 3000 within 60–90 seconds (ClickHouse first-boot initialisation takes the longest).
Open the web UI and create your first project
Navigate to https://your-domain.com. Langfuse shows the signup screen on first boot. Create your admin account, then go to Settings → Projects → Create project. Copy the Public Key and Secret Key from the project settings — you'll need them in your application.
Instrument your Python or TypeScript application
In Python: pip install langfuse, set LANGFUSE_HOST=https://your-domain.com, LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY, then decorate your LLM calls with @observe() or use langfuse.trace(). In TypeScript: npm install langfuse, initialise the client with your host and keys. Traces appear in the dashboard within seconds of the first call.
Enable automatic tracing if you use LiteLLM
If LiteLLM is already in your stack (which it likely is if you use the ServOrbit AI stack), add success_callback = ["langfuse"] to litellm_config.yaml and set the three Langfuse env vars. Every proxied LLM call — across all downstream models — is traced automatically, with cost and token data attached.
Run your first evaluation
Go to Traces, filter for a representative sample, click 'Add to dataset'. Open Datasets → your dataset → Run evaluation, choose LLM-as-a-judge with a prompt template ('Rate the relevance of this response on a scale of 1–5'), and submit. Scores appear on every trace in the set and feed into the aggregate analytics dashboard.
Logging in for the first time
The URL opens the Langfuse screen with a “Sign up” link: create your account, then your organisation and your first project in the wizard.
Pair with LiteLLM for complete AI cost visibility
The ServOrbit AI stack already includes LiteLLM as an OpenAI-compatible gateway. Connect Langfuse to LiteLLM with three env vars and you get a complete picture: LiteLLM enforces rate limits and routes between providers; Langfuse records every trace with full prompt, completion, model, tokens and cost. Together they give you a private, auditable AI operations layer — no SaaS middleman, no data egress.
LLM costs showing $0.00 in Tracing v4 — what is happening and how to work around it
An active bug affects Langfuse’s Tracing v4 view (tracked in issue #16077, 14 comments in August 2026): the Cost ($) column always displays $0.00 for every trace.
The root cause is structural. The v4 view reads the cost stored on the root observation of each trace, which is of type SPAN — a span carries no cost of its own; it only wraps child generation observations. The actual cost is distributed across child GENERATION observations, but the view does not aggregate them: it reads the root, finds zero, and displays zero.
Your data is intact. Cost is correctly recorded on each generation; only the display in the main view is wrong. Three workarounds are available while the official fix is pending:
1. Analytics / Metrics tab — the aggregate view computes costs across all generations, not just the root. This is the reliable source for tracking your daily spend.
2. Filter by GENERATION type in Traces — add the filter observation_type = GENERATION. Every row shows its real cost.
3. Open the trace detail panel — the side panel shows the full breakdown by span and generation with correct costs. Useful to diagnose a specific trace.
If you observe this on your instance, confirm you are running Langfuse v3.x with Tracing v4 enabled. The issue is open and tracked by the Langfuse team.
Langfuse vs Langsmith vs Helicone
Langsmith (LangChain's hosted product) and Helicone are the main SaaS alternatives. Both are excellent — and both require you to route your traces through their servers. For teams handling sensitive prompts (legal, financial, medical), or for compliance environments that prohibit third-party data egress, self-hosting is not optional. Langfuse gives you the same feature set (tracing, evals, prompt management, cost analytics) on infrastructure you control. The MIT licence also means you can read, modify and audit every line of the code your traces flow through.
