Why an MLflow server
Run locally, MLflow writes its results to an mlruns folder on your machine: nobody else sees them, and they disappear with the machine. A tracking server changes that. Every notebook, every training script and every team member sends their runs to the same place, and the interface compares them side by side.
The same server holds the model registry — the list of validated, versioned models, with an alias for the one in production — and, for generative AI, the traces of your calls to language models.
What MLflow brings
- Experiment tracking — parameters, metrics and artifacts of every run, filtered and compared in the interface.
- Model registry — versions, aliases and history, so you know exactly what runs in production.
- LLM tracing — inputs, outputs, latency and steps of every request of your generative AI applications.
- Artifacts served by the server — files, charts and models uploaded over HTTPS, no S3 storage to configure.
- Automatic logging —
autolog()for scikit-learn, PyTorch, Transformers, XGBoost, OpenAI or LangChain. - Accounts and permissions — one account per person, permissions granted experiment by experiment.
MLflow versus tracking by hand
Scroll the table
| MLflow server | Notebooks and spreadsheet | |
|---|---|---|
| Run history | Every run recorded automatically | Whatever someone remembered to write down |
| Comparing two models | One screen, metrics side by side | Copying numbers |
| Produced files | Stored with the run that created them | Scattered across laptops |
| Model in production | A version and an alias in the registry | A file named "final_v3" |
| Teamwork | A shared server, accounts | Sending files around |
Prerequisites
An Ubuntu 24.04 VPS with 2 GB of RAM is enough to start; plan 4 GB and more disk if you store many models, since artifacts take up the space. You need Docker with the Compose plugin, and a domain name pointing to the VPS, with nginx and an HTTPS certificate: your scripts and your team will connect to it.
Install MLflow with Docker Compose
Prepare the directory and the secrets
Create the application directory and a
.envfile holding theadminaccount's password, the session key and your domain.mkdir -p /opt/mlflow && cd /opt/mlflow cat > .env <<EOF ADMIN_PASSWORD=$(openssl rand -hex 16) SECRET_KEY=$(openssl rand -hex 24) DOMAIN=mlflow.example.ma EOF chmod 600 .envWrite the docker-compose.yml
The
-fullimage is the official variant that ships the authentication module. At startup, the command writes thebasic_auth.inifile that declares theadminaccount;--allowed-hostsonly accepts your domain and the loopback. The port is only published on the loopback: only nginx reaches it.MLFLOW_SERVER_ENABLE_JOB_EXECUTION=falseturns off the background job runner, which is memory-hungry and not needed for tracking.services: mlflow: image: ghcr.io/mlflow/mlflow:v3.18.0-full restart: unless-stopped ports: - "127.0.0.1:5000:5000" environment: ADMIN_PASSWORD: ${ADMIN_PASSWORD} DOMAIN: ${DOMAIN} MLFLOW_FLASK_SERVER_SECRET_KEY: ${SECRET_KEY} MLFLOW_AUTH_CONFIG_PATH: /mlflow/basic_auth.ini MLFLOW_SERVER_ENABLE_JOB_EXECUTION: "false" entrypoint: ["/bin/sh", "-c"] command: - >- mkdir -p /mlflow/artifacts && printf '[mlflow]\ndefault_permission = NO_PERMISSIONS\ndatabase_uri = sqlite:////mlflow/basic_auth.db\nadmin_username = admin\nadmin_password = %s\nauthorization_function = mlflow.server.auth:authenticate_request_basic_auth\n' "$$ADMIN_PASSWORD" > /mlflow/basic_auth.ini && exec mlflow server --app-name basic-auth --host 0.0.0.0 --port 5000 --workers 2 --backend-store-uri sqlite:////mlflow/mlflow.db --artifacts-destination /mlflow/artifacts --allowed-hosts "$$DOMAIN,localhost:*,127.0.0.1:*" volumes: - mlflow-data:/mlflow volumes: mlflow-data: {}Start and check
Startup takes about fifteen seconds.
/healthanswers without authentication; the API returns 401 without credentials.docker compose up -d curl -fsS http://127.0.0.1:5000/health curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5000/api/2.0/mlflow/experiments/searchPublish over HTTPS with nginx
In your domain's vhost (certificate obtained with certbot), proxy to the local port and pass the
Hostheader. Raiseclient_max_body_size: a model uploaded as an artifact quickly exceeds the default limit.location / { proxy_pass http://127.0.0.1:5000; proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto $scheme; client_max_body_size 512m; }
Without --app-name basic-auth, the server is open
By default, mlflow server has no authentication: anyone who reaches the address can read your experiments, download your models and delete everything. Keep --app-name basic-auth, never publish port 5000 on the public IP, and keep --allowed-hosts set to your domain only: it is MLflow's protection against DNS rebinding.
Send your experiments from Python
Install the client (pip install mlflow) on the training machine, then point it to your server. Credentials are passed through environment variables:
import os
import mlflow
os.environ["MLFLOW_TRACKING_USERNAME"] = "admin"
os.environ["MLFLOW_TRACKING_PASSWORD"] = "YOUR_PASSWORD"
mlflow.set_tracking_uri("https://mlflow.example.ma")
mlflow.set_experiment("churn-model")
with mlflow.start_run():
mlflow.log_param("learning_rate", 0.01)
mlflow.log_metric("accuracy", 0.93)
mlflow.log_artifact("confusion_matrix.png")The run appears in the interface right away, with its parameters, metrics and the uploaded file. mlflow.sklearn.autolog() or mlflow.pytorch.autolog() record all of that without an extra line.
Trace an LLM application
For an assistant or an agent, tracing records every request with its model calls, its steps and their duration. Turn on your library's automatic logging, or decorate your own functions:
import mlflow
mlflow.set_experiment("support-assistant")
mlflow.openai.autolog() # every OpenAI client call becomes a trace
@mlflow.trace
def answer(question: str) -> str:
...Traces show up in the experiment's tab: you see which call was slow, which context was sent and what the model answered.
The model registry
When a run is validated, register its model: it gets a version number, and you can give it an alias such as production.
mlflow.register_model(f"runs:/{run_id}/model", "churn-model")Your inference service then loads models:/churn-model@production: switching models means moving the alias, without touching the code.
Accounts and permissions
The admin account creates the other accounts through the API:
curl -u admin:YOUR_PASSWORD -X POST https://mlflow.example.ma/api/2.0/mlflow/users/create \
-H 'Content-Type: application/json' \
-d '{"username": "sara", "password": "A_LONG_PASSWORD"}'A new account sees no experiment until one is shared with it: permissions (read, edit, manage) are granted experiment by experiment and model by model. Change the admin password through the users/update-password API (passing the old one in current_password); the new one is kept across restarts.
Backup and updates
All of MLflow's state lives in the mlflow-data volume: the experiment database, the accounts database and the artifacts folder. Back it up with the container stopped, then copy the archive off the VPS:
cd /opt/mlflow
docker compose stop
docker run --rm -v mlflow_mlflow-data:/data -v "$PWD":/backup alpine \
tar czf /backup/mlflow-$(date +%F).tar.gz -C /data .
docker compose startTo update, read the release notes first — an update can migrate the database schema —, change the image tag in docker-compose.yml, then:
cd /opt/mlflow
docker compose pull && docker compose up -d
docker compose logs --tail 201-click install with ServOrbit
ServOrbit's 1-click apps install MLflow on your VPS without a command line: it deploys the official container, generates the admin account's password and the session key, enables authentication on the interface and the API, limits the accepted host names to your address and publishes the server over HTTPS on a subdomain of your domain or on the VPS's free subdomain (mlflow.{dns_slug}.servorbit-dns.com). The password is shown in your client area; all that is left is to connect your notebooks.
