[{"data":1,"prerenderedAt":108},["ShallowReactive",2],{"seo-verification":3,"marketplace-app-en-airflow":6},{"google":4,"bing":5},"EycwPY2XMyTkVzas3n1ygeNJFGAH513qrMjfDljzsMQ","",{"key":7,"data":8},"marketplace-app-en-airflow",{"slug":9,"slugs":10,"categorySlugs":11,"name":16,"description":17,"phase":18,"unavailableReason":19,"docsUrl":20,"logo":21,"github":22,"tagline":23,"longDescription":24,"features":25,"useCases":32,"steps":42,"faq":58,"specs":74,"compatibleOs":83,"relatedApps":84,"relatedPosts":103,"category":105},"airflow",{"fr":9,"en":9,"ar":9,"es":9},{"fr":12,"en":13,"ar":14,"es":15},"automatisation","automation-workflows","الأتمتة-وسير-العمل","automatizacion","Apache Airflow","Open-source Python data workflow orchestrator: schedule and monitor your ETL DAGs and data pipelines from the web UI, with a robust scheduler and 600+ providers. LocalExecutor on your VPS, no Celery required.",2,"awaiting_qa","https:\u002F\u002Fservorbit.com\u002Fblog\u002Finstalling-apache-airflow-on-a-vps-self-hosted-automation","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F47359","https:\u002F\u002Fgithub.com\u002Fapache\u002Fairflow","Self-hosted Python DAG orchestrator — ETL, data pipelines and scheduling with 600+ providers on your VPS.","Apache Airflow is the de facto standard for orchestrating data pipelines as directed acyclic graphs (DAGs) written in Python. Each DAG is a plain .py file that defines tasks, their dependencies, and their schedule. The scheduler picks up DAG files automatically, executes tasks across the configured workers, and exposes a full-featured web UI for monitoring runs, inspecting logs, and managing connections.\n\nDeployed on a ServOrbit VPS with LocalExecutor, Airflow runs three containers — webserver, scheduler and PostgreSQL — without Celery or Redis. This lighter stack consumes around 1–2 GB of RAM at idle, well within reach of a mid-range VPS, while supporting hundreds of tasks per day. Your DAGs run as close as possible to your internal databases and APIs, and all pipeline metadata stays on your infrastructure.",[26,27,28,29,30,31],"Python DAGs versioned in Git — tasks, dependencies and schedules in one .py file","600+ provider packages — SQL, cloud (AWS, GCP, Azure), dbt, Spark, HTTP and more","Robust scheduler — cron, dependencies between tasks, backfill and catchup","Full-featured web UI — DAG graph, task logs, run history and variable management","LocalExecutor mode — 3 containers (webserver + scheduler + PostgreSQL), no Celery\u002FRedis","Apache-2.0 licence — 36 k+ GitHub stars, active community, no vendor lock-in",[33,36,39],{"title":34,"body":35},"ETL and data pipelines","Describe your data ingestion, transformation and export jobs as Python DAGs. Airflow handles scheduling, retries, dependency resolution and alerting on failure — leaving you to write business logic, not orchestration plumbing.",{"title":37,"body":38},"Scheduled data science workflows","Run Jupyter notebooks, dbt models or ML training scripts on a schedule. Airflow's sensors let each downstream step wait for its upstream data to be ready before executing, without polling loops.",{"title":40,"body":41},"Internal process automation","Automate any repeatable multi-step process — report generation, database maintenance, API syncs — without paying for a cloud scheduler. LocalExecutor handles hundreds of tasks per day on a standard VPS.",[43,46,49,52,55],{"title":44,"body":45},"Order a VPS","Minimum 2 vCPU and 4 GB RAM for LocalExecutor with a moderate task load. Ubuntu 24.04 with Docker is the recommended OS. Plan for at least 20 GB of disk for logs and the Airflow database.",{"title":47,"body":48},"Deploy from the Marketplace","Marketplace → Automation & Workflows → Apache Airflow → Deploy. Three containers start: PostgreSQL (database), airflow-init (runs migrations and creates the admin account, then exits), then webserver and scheduler. The webserver listens on port 8080, bound to 127.0.0.1.",{"title":50,"body":51},"Log in for the first time","The credentials are displayed in your ServOrbit client area after the deployment. Open an SSH tunnel: `ssh -L 8080:127.0.0.1:\u003Cport> root@\u003Cvps-ip>`, then open `http:\u002F\u002Flocalhost:8080` in your browser. For a public URL, attach a domain in your client area — nginx will reverse-proxy to Airflow and provision a TLS certificate automatically.",{"title":53,"body":54},"Create your first DAG","DAG files live in the `dags` volume mounted at `\u002Fopt\u002Fairflow\u002Fdags`. Create a `.py` file in that directory: the scheduler detects it in a few seconds and displays it in the web UI. Enable it with the toggle and trigger a manual run to verify it works.",{"title":56,"body":57},"Monitor and maintain","The web UI shows each DAG's run history, task durations, logs and failure reasons. Set up email alerts with `AIRFLOW__EMAIL__EMAIL_CONN_ID` and a configured SMTP connection, or use the Slack provider to notify your team.",[59,62,65,68,71],{"q":60,"a":61},"Does Apache Airflow require a GPU?","No. Airflow is a workflow orchestrator, not a compute engine. It schedules and monitors Python tasks on CPU — the tasks themselves can call GPU APIs or Spark clusters, but Airflow only needs standard RAM and CPU.",{"q":63,"a":64},"What is the difference between LocalExecutor and CeleryExecutor in Airflow?","LocalExecutor runs tasks in parallel as local processes on the same server as the scheduler — ideal for hundreds of tasks per day on a single VPS, no Redis required. CeleryExecutor distributes tasks across remote workers — needed for thousands of concurrent tasks or multiple servers. For typical use on a ServOrbit VPS, LocalExecutor is the recommendation.",{"q":66,"a":67},"How much RAM does Apache Airflow need on a VPS?","In LocalExecutor mode, expect 1–2 GB of RAM at idle (webserver + scheduler + PostgreSQL). For Python DAGs loading pandas or large files, plan for 4 GB to have headroom for task processes. CeleryExecutor mode — with workers and Redis — requires 4–8 GB depending on the number of workers.",{"q":69,"a":70},"How do I access Airflow without a domain name?","Open an SSH tunnel: `ssh -L 8080:127.0.0.1:\u003Cport> root@\u003Cvps-ip>` (replace `\u003Cport>` with the port shown in your client area), then open `http:\u002F\u002Flocalhost:8080`. For a permanent public URL, attach a domain in your client area — nginx will reverse-proxy to Airflow and provision a TLS certificate automatically.",{"q":72,"a":73},"Can I add custom Python dependencies to Airflow?","Yes. Build a Docker image derived from `apache\u002Fairflow:3.3.0` with a `requirements.txt` file: add a `FROM apache\u002Fairflow:3.3.0` line and then `RUN pip install --no-cache-dir your-package`. Use this image in your compose instead of the official image. This is the recommended method — installing packages at runtime is slow and not reproducible.",{"ram":75,"cpu":76,"disk":77,"stack":78,"port":82},"4 GB","2 vCPU","20 GB",[79,80,81],"Docker","Python 3.12","PostgreSQL 16","8080",[],[85,92,98],{"name":86,"slug":86,"categorySlug":13,"categoryName":87,"categoryColor":88,"logo":89,"tagline":90,"description":91},"n8n","Automation & Workflows","text-brand-action bg-brand-action\u002F10","https:\u002F\u002Fcdn.simpleicons.org\u002Fn8n","Automate your business processes with 400+ integrations — 100% self-hosted on your VPS.","Automate your workflows across applications without code. Over 400 integrations, 100% self-hosted on your VPS.",{"name":93,"slug":94,"categorySlug":13,"categoryName":87,"categoryColor":88,"logo":95,"tagline":96,"description":97},"Kestra","kestra",null,"Declarative YAML orchestration for data pipelines and ETL — self-hosted on your VPS, Apache-2.0, 15 k+ stars.","Declarative YAML workflow orchestration platform — self-hosted data pipelines, ETL and event-driven automation. Orchestrate your internal processing tasks, triggered by schedule, webhook or API.",{"name":99,"slug":100,"categorySlug":13,"categoryName":87,"categoryColor":88,"logo":95,"tagline":101,"description":102},"Windmill","windmill","Run your scripts as workflows, APIs and UIs — 100% self-hosted on your VPS.","Code-first self-hosted automation platform: turn Python, TypeScript, Go or Bash scripts into workflows, internal APIs and UIs — 17k ⭐, AGPLv3.",[104],"installer-apache-airflow-vps",{"key":12,"slug":13,"name":87,"objective":106,"icon":107,"color":88},"Automate business processes.","automation",1790253312511]