Why self-host Kestra on a VPS
Kestra stands out for its declarative approach: a workflow (flow) is a versionable YAML file, with chained tasks, conditions, loops and triggers. This Everything-as-Code approach appeals to data teams that want Git, code review and reproducibility, without the heaviness of writing Python DAGs. Self-hosting Kestra on a VPS makes complete sense for pipelines that read from and write to your databases: the processing runs where your data lives, with no external transit. Kestra can also launch tasks in isolated Docker containers, which makes a VPS with an accessible Docker daemon particularly suitable. The web interface offers a flow editor, a topological view and complete execution tracking.
Concrete benefits of self-hosting
- Workflows in YAML versioned in Git, code review and total reproducibility.
- Rich plugins: SQL databases, S3, Python/Bash scripts, Docker containers, HTTP requests.
- Varied triggers: cron, event, webhook, file detection.
- Execution as close as possible to internal databases, with no external transit.
- Web interface with a visual editor, topological view and detailed execution logs.
- No cost per execution: only your VPS limits the volume of processing.
Technical requirements
Kestra needs PostgreSQL as a backend (for the simplest version, it uses an internal database, but PostgreSQL is recommended in production). Plan for at least 2 vCPU and 4 GB of RAM, because Kestra's JVM consumes memory; aim for 8 GB if your flows launch Docker tasks or heavy data scripts. Allow 30 GB of disk, Docker and Docker Compose, a domain (orchestrator.yourdomain.com). Important point: for Kestra to launch tasks in containers, mount the Docker socket (/var/run/docker.sock) inside the Kestra container.
Deploy Kestra with Docker Compose
Prepare the VPS
Install Docker (curl -fsSL https://get.docker.com | sh) and create /opt/kestra. Check the available RAM: the JVM loves memory, plan for at least 4 GB genuinely free, using free -h.
Write the docker-compose
Fetch Kestra's official compose, which includes PostgreSQL. Declare the kestra service with the kestra/kestra:latest image, the server standalone command, and mount /var/run/docker.sock:/var/run/docker.sock to enable containerized tasks.
Configure the PostgreSQL backend
In the configuration section of the compose, point kestra.datasources to PostgreSQL with the JDBC URL, the user and the password. Also set kestra.url: https://orchestrator.yourdomain.com/ for correct links in the UI.
Start Kestra
Run docker compose up -d. On first startup, Kestra creates its schema in PostgreSQL. Monitor docker compose logs -f kestra until the message indicating the server is ready on port 8080.
Reverse proxy and HTTPS
Put Caddy in front: orchestrator.yourdomain.com { reverse_proxy localhost:8080 }. HTTPS protects access to the flow editor and the execution logs, which may contain sensitive data.
Create a first YAML flow
In the UI, create a test flow with an io.kestra.plugin.core.log.Log task then add a cron schedule trigger. Confirm that a scheduled execution launches and appears in the timeline. Then version this YAML in Git.
Explicitly set the JVM memory to avoid surprises on a VPS: pass JAVA_OPTS=-Xmx2g (or half of your RAM) in the Kestra container's environment. Without this limit, the JVM may attempt to reserve too much heap and trigger an OOM kill by the Linux kernel. For data pipelines, use the Docker task type rather than Process: each processing task runs in its own container with its dependencies, which avoids polluting the VPS and guarantees reproducibility.
