Those resources and services are not maintained, nor endorsed by the Apache Airflow Community and Apache Airflow project (maintained by the Committers and the Airflow PMC). Use them at your sole discretion. The community does not verify the licences nor validity of those tools, so it's your responsibility to verify them.
If you would you like to be included on this page, please reach out to the Apache Airflow dev or user mailing list and let us know or simply open a Pull Request to that page.
Learning resources
Apache Airflow YouTube Channel - Official YouTube Channel
Airflow Summit - Online conference for Apache Airflow developers
Cwl-airflow Last Built. 3 weeks, 6 days ago passed. CWL-Airflow is one of the first pipeline managers supporting version 1.0 of the CWL standard and provides a robust and user-friendly interface for executing CWL pipelines. Unlike more complicated pipeline managers, the installation of Airflow and the CWL-Airflow extension can be performed with a single pip install command. Apache/airflow: Apache Airflow - A platform to, Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows. Flow logs can publish flow log data directly to Amazon CloudWatch. When publishing to CloudWatch Logs, flow log data is published to a log group, and each network interface has a unique log stream in the log group.
Awesome Apache Airflow - Curated list of resources about Apache Airflow
The Complete Hands-On Introduction to Apache Airflow by Marc Lamberti on Udemy
Apache Airflow: Complete Hands-On Beginner to Advanced Class by Alexandra Abbas on Udemy
Airflow as a Service
Astronomer - Managed Apache Airflow in Astronomer Cloud, or self-hosted within your environment
Google Cloud Composer - Managed Apache Airflow service on Google Cloud Platform
Qubole - Managed Apache Airflow Service on all major public clouds
Amazon Managed Workflows for Apache Airflow - Managed Apache Airflow on Amazon Web Services (AWS)
Third Party Airflow Plugins and Providers
Astronomer Registry - The discovery and distribution hub for Apache Airflow integrations created to aggregate and curate the best bits of the ecosystem.
Airflow Plugins - Central collection of repositories of various plugins for Airflow, including mailchimp, trello, sftp, GitHub, etc.
Airflow ECR Plugin - Plugin to refresh AWS ECR login token at regular intervals. This is helpful where DockerOperator needs to pull images hosted on ECR.
Tools integrating with Airflow
afctl - A CLI tool that includes everything required to create, manage and deploy airflow projects faster and smoother.
Cwl Airflow Controller
airflow-aws-executors - Run Airflow Tasks directly on AWS Batch, AWS Fargate, or AWS ECS; provisioning less infra is more.
Astronomer - Managed Apache Airflow in Astronomer Cloud, or self-hosted within your environment
Google Cloud Composer - Managed Apache Airflow service on Google Cloud Platform
Qubole - Managed Apache Airflow Service on all major public clouds
Amazon Managed Workflows for Apache Airflow - Managed Apache Airflow on Amazon Web Services (AWS)
Third Party Airflow Plugins and Providers
Astronomer Registry - The discovery and distribution hub for Apache Airflow integrations created to aggregate and curate the best bits of the ecosystem.
Airflow Plugins - Central collection of repositories of various plugins for Airflow, including mailchimp, trello, sftp, GitHub, etc.
Airflow ECR Plugin - Plugin to refresh AWS ECR login token at regular intervals. This is helpful where DockerOperator needs to pull images hosted on ECR.
Tools integrating with Airflow
afctl - A CLI tool that includes everything required to create, manage and deploy airflow projects faster and smoother.
Cwl Airflow Controller
airflow-aws-executors - Run Airflow Tasks directly on AWS Batch, AWS Fargate, or AWS ECS; provisioning less infra is more.
airflow-code-editor - A tool for Apache Airflow that allows you to edit DAGs in browser.
airflow-diagrams - Auto-generated Diagrams from Airflow DAGs
airflow-maintenance-dags - Clairvoyant has a repo of Airflow DAGs that operator on Airflow itself, clearing out various bits of the backing metadata store.
AirflowK8sDebugger - A library for generate k8s pod yaml templates from an Airflow dag using the KubernetesPodOperator.
Airflow Ditto - An extensible framework to do transformations to an Airflow DAG and convert it into another DAG which is flow-isomorphic with the original DAG, to be able to run it on different environments (e.g. on different clouds, or even different container frameworks - Apache Spark on YARN vs Kubernetes). Comes with out-of-the-box support for EMR-to-HDInsight-DAG transforms.
Apache-Liminal-Incubating - Liminal provides a domain-specific-language (DSL) to build ML/AI workflows on top of Apache Airflow. Its goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production.
Cwl Airflow System
Chartis - Python package to convert Common Workflow Language (CWL) into Airflow DAG.
CWL-Airflow - Python package to extend Apache-Airflow 1.10.11 functionality with CWL v1.2 support.
dag-factory - A library for dynamically generating Apache Airflow DAGs from YAML configuration files.
Dag Dependencies viewer - A tool which creates a view to visualize dependencies between the Airflow DAGs
Databand - Observability platform built on top of Airflow.
dbt (data build tool) - Data transformation tool, dbt jobs can be scheduled using Airflow.
GeniumCloud - One-Stop-Shop Platform for rapid build, scheduling and control Airflow workflows via completely new UI. Out of the box comprehensive Airflow infrastructure monitoring, integration with alerting systems and service adoption from small to enterprise organizations. The easiest way to manage complex workflows.
gusty - Create a DAG using any number of YAML, Python, Jupyter Notebook, or R Markdown files that represent individual tasks in the DAG. gusty also configures dependencies, DAGs, and TaskGroups, features support for your local operators, and more. A fully containerized demo is available here.
Meltano - Open source, self-hosted, CLI-first, debuggable, and extensible ELT tool that embraces Singer for extraction and loading, leverages dbt for transformation, and integrates with Airflow for orchestration.
Oozie to Airflow - A tool to easily convert between Apache Oozie workflows and Apache Airflow workflows.
Pylint-Airflow - A Pylint plugin for static code analysis on Airflow code.
Cwl Airflow Filter
simple-dag-editor - Zero configuration Airflow tool that let you manage your DAG files.
Viewflow - An Airflow-based framework that allows data scientists to create data models without writing Airflow code.
whirl - Fast iterative local development and testing of Apache Airflow workflows.
