# Apache Airflow

> Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows using Python.

Apache Airflow is a community-driven, open-source workflow orchestration platform that lets data engineers and developers define pipelines as Python code. It provides a modular, scalable architecture built around a message queue that can orchestrate an arbitrary number of workers. Airflow's web UI offers real-time monitoring, scheduling, and management of all workflows, while its extensive provider ecosystem enables plug-and-play integrations with virtually every major cloud and data platform.

- **Pure Python Pipelines**: *Define DAGs (Directed Acyclic Graphs) entirely in Python — no XML or command-line magic required — enabling dynamic pipeline generation with loops, conditionals, and date-time scheduling.*
- **Scalable Architecture**: *Airflow's modular design uses a message queue to distribute work across an arbitrary number of workers, scaling from a single machine to large distributed clusters.*
- **Robust Web UI**: *Monitor, schedule, and manage workflows through a modern web application that provides full visibility into task status, logs, and historical runs.*
- **Extensible Operator Library**: *Easily define custom operators and extend existing libraries to match your environment's abstraction level.*
- **Jinja Templating**: *Parametrize pipelines elegantly using the built-in Jinja templating engine for clean, explicit pipeline definitions.*
- **Helm Chart & Docker Support**: *Deploy Airflow on Kubernetes using the official Helm Chart, or use the official Docker image for containerized deployments.*
- **Task SDK**: *Decouple DAG authoring from Airflow internals with the Task SDK, providing a stable, forward-compatible interface for writing tasks across Airflow versions.*
- **Provider Packages**: *Access 90+ independently versioned provider packages for integrations with Google Cloud, AWS, Azure, Snowflake, Databricks, dbt Cloud, Kafka, Spark, and many more.*
- **REST API & Python Client**: *Interact with Airflow programmatically via the official REST API and Python API client.*
- **Active Open-Source Community**: *Contribute via GitHub pull requests with no barriers; join a large Slack community for support and knowledge sharing.*

## Features
- Python-based DAG authoring
- Dynamic pipeline generation
- Web UI for monitoring and scheduling
- Modular scalable architecture
- Jinja templating engine
- Helm Chart for Kubernetes deployment
- Official Docker image
- Task SDK for decoupled DAG authoring
- REST API
- Python API client
- 90+ provider packages
- Plug-and-play cloud integrations
- Custom operator support
- Message queue-based worker orchestration
- Task logging and status tracking

## Integrations
Google Cloud Platform, Amazon Web Services, Microsoft Azure, Apache Spark, Apache Kafka, Snowflake, Databricks, dbt Cloud, PostgreSQL, MySQL, MongoDB, Redis, Elasticsearch, Slack, Docker, Kubernetes, Salesforce, Tableau, OpenAI, Pinecone, Weaviate, Airbyte, GitHub, Jenkins, Datadog, PagerDuty, Sendgrid, Telegram, Trino, Presto

## Platforms
WINDOWS, WEB, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Links
- Website: https://airflow.apache.org
- Documentation: https://airflow.apache.org/docs/apache-airflow/stable/index.html
- Repository: https://github.com/apache/airflow
- EveryDev.ai: https://www.everydev.ai/tools/apache-airflow