What is the difference between LocalExecutor and CeleryExecutor in Airflow?

LocalExecutor runs tasks in parallel as local processes on the same server as the scheduler — ideal for hundreds of tasks per day on a single VPS, no Redis required. CeleryExecutor distributes tasks across remote workers — needed for thousands of concurrent tasks or multiple servers. For typical use on a ServOrbit VPS, LocalExecutor is the recommendation.

How much RAM does Apache Airflow need on a VPS?

In LocalExecutor mode, expect 1–2 GB of RAM at idle (webserver + scheduler + PostgreSQL). For Python DAGs loading pandas or large files, plan for 4 GB to have headroom for task processes. CeleryExecutor mode — with workers and Redis — requires 4–8 GB depending on the number of workers.

Automate business processes.

Apache Airflow

Self-hosted Python DAG orchestrator — ETL, data pipelines and scheduling with 600+ providers on your VPS.

4 GB RAM 2 vCPU Port 8080 In validation

In validation

Automatic installation for this solution is ready and currently going through our tests on a real server. Ordering will open as soon as validation is complete.

Source code

Tech stack

DockerPython 3.12PostgreSQL 16

Minimum RAM4 GB

Minimum CPU2 vCPU

Default port8080

Compatible OSChoice of Linux distributions

Apache Airflow is the de facto standard for orchestrating data pipelines as directed acyclic graphs (DAGs) written in Python. Each DAG is a plain .py file that defines tasks, their dependencies, and their schedule. The scheduler picks up DAG files automatically, executes tasks across the configured workers, and exposes a full-featured web UI for monitoring runs, inspecting logs, and managing connections.

Deployed on a ServOrbit VPS with LocalExecutor, Airflow runs three containers — webserver, scheduler and PostgreSQL — without Celery or Redis. This lighter stack consumes around 1–2 GB of RAM at idle, well within reach of a mid-range VPS, while supporting hundreds of tasks per day. Your DAGs run as close as possible to your internal databases and APIs, and all pipeline metadata stays on your infrastructure.

Key features

Python DAGs versioned in Git — tasks, dependencies and schedules in one .py file

600+ provider packages — SQL, cloud (AWS, GCP, Azure), dbt, Spark, HTTP and more

Robust scheduler — cron, dependencies between tasks, backfill and catchup

Full-featured web UI — DAG graph, task logs, run history and variable management

LocalExecutor mode — 3 containers (webserver + scheduler + PostgreSQL), no Celery/Redis

Apache-2.0 licence — 36 k+ GitHub stars, active community, no vendor lock-in

When to use this solution?

1

ETL and data pipelines

Describe your data ingestion, transformation and export jobs as Python DAGs. Airflow handles scheduling, retries, dependency resolution and alerting on failure — leaving you to write business logic, not orchestration plumbing.

2

Scheduled data science workflows

Run Jupyter notebooks, dbt models or ML training scripts on a schedule. Airflow's sensors let each downstream step wait for its upstream data to be ready before executing, without polling loops.

3

Internal process automation

Automate any repeatable multi-step process — report generation, database maintenance, API syncs — without paying for a cloud scheduler. LocalExecutor handles hundreds of tasks per day on a standard VPS.

Deploy Apache Airflow on your VPS

Guide optimized for ServOrbit Cloud VPS.

01

Order a VPS

Minimum 2 vCPU and 4 GB RAM for LocalExecutor with a moderate task load. Ubuntu 24.04 with Docker is the recommended OS. Plan for at least 20 GB of disk for logs and the Airflow database.

02

Deploy from the Marketplace

Marketplace → Automation & Workflows → Apache Airflow → Deploy. Three containers start: PostgreSQL (database), airflow-init (runs migrations and creates the admin account, then exits), then webserver and scheduler. The webserver listens on port 8080, bound to 127.0.0.1.

03

Log in for the first time

The credentials are displayed in your ServOrbit client area after the deployment. Open an SSH tunnel: ssh -L 8080:127.0.0.1:<port> root@<vps-ip>, then open http://localhost:8080 in your browser. For a public URL, attach a domain in your client area — nginx will reverse-proxy to Airflow and provision a TLS certificate automatically.

04

Create your first DAG

DAG files live in the dags volume mounted at /opt/airflow/dags. Create a .py file in that directory: the scheduler detects it in a few seconds and displays it in the web UI. Enable it with the toggle and trigger a manual run to verify it works.

05

Monitor and maintain

The web UI shows each DAG's run history, task durations, logs and failure reasons. Set up email alerts with AIRFLOW__EMAIL__EMAIL_CONN_ID and a configured SMTP connection, or use the Slack provider to notify your team.

Frequently asked questions

No. Airflow is a workflow orchestrator, not a compute engine. It schedules and monitors Python tasks on CPU — the tasks themselves can call GPU APIs or Spark clusters, but Airflow only needs standard RAM and CPU.

Open an SSH tunnel: ssh -L 8080:127.0.0.1:<port> root@<vps-ip> (replace <port> with the port shown in your client area), then open http://localhost:8080. For a permanent public URL, attach a domain in your client area — nginx will reverse-proxy to Airflow and provision a TLS certificate automatically.

Yes. Build a Docker image derived from apache/airflow:3.3.0 with a requirements.txt file: add a FROM apache/airflow:3.3.0 line and then RUN pip install --no-cache-dir your-package. Use this image in your compose instead of the official image. This is the recommended method — installing packages at runtime is slow and not reproducible.

Complementary solutions

Automation & Workflows

Apache Airflow isn't orderable yet.