Airflow
Airflow is a platform to programmatically author, schedule and monitor workflows. Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. Airflow Developments A wide range of air flow products for all your needs. Your basket is empty Products. Intermittent Extractor Fans. Continuous Running Extractor Fans. Mechanical Ventilation With Heat Recovery. Industrial Fans. Airflow is a platform created by the community to programmatically author, schedule, and monitor workflows. Machine learning is the hot topic of the industry. It won't be so cool if not for the data processing involved Airflow is an ETL (Extract, Transform, Load) workflow orchestration tool, used in data transformation pipelines.
Airflow is building the first electric Short Take-Off and Landing (eSTOL) aircraft designed for sustainable same-day e-commerce.
Airflow’s eSTOL aircraft can move cargo quickly and cost-effectively by flying over traffic.
Logistics and e-commerce companies can:
- Lower inventory carrying costs by having a way to consolidate inventory at the regional level
- Offer more SKUs for same-day delivery by moving goods into cities from regional distribution centers quickly and inexpensively
- Expand same-day and next-day service to more areas
- Save 632,000 lbs of CO2 per aircraft annually over today’s methods
Airflow Appliance Repair
The logistics industry is undergoing unprecedented changes. In order to support same-day and next-day e-commerce shipments, the industry is moving from a central hub and spoke model to a regional distribution model.
Airflow’s aircraft and services provide for the rapid movement of goods throughout a region, enabling widely expanded and low-cost same-day delivery.
Airflow Tutorial
The need for rapid middle-mile logistics is growing due to e-commerce growth and increasing road congestion. The current model is very inefficient because lots of inventory must be stocked close-in to cities in hundreds of delivery centers to allow for same-day delivery. This drives high inventory carrying costs and limits what can be offered to consumers.
A solution is to store inventory in regional warehouses and then move inventory to last-mile delivery centers quickly and inexpensively. Airflow provides that solution.
Airflow offers the first aerial logistics service that can move cargo directly between regional warehouses and delivery centers without the use of traditional airports.
Conventional aircraft need airports with runways that are thousands of feet long. Airflow eSTOL aircraft require only a few hundred feet for takeoff and landing (about the length of a football field). That means runways can be built almost anywhere, at regional distribution centers and local delivery centers using existing regulations.
Our solution is simple: use aircraft to move goods rather than stuffing delivery centers full of inventory. With Airflow, goods can be stocked in large regional distribution centers and quickly moved to smaller delivery centers for last-mile delivery.
| Original author(s) | Maxime Beauchemin / Airbnb |
|---|---|
| Developer(s) | Apache Software Foundation |
| Initial release | June 3, 2015; 5 years ago |
| Stable release | 2.0.1 (February 8, 2021; 2 months ago[1]) [±] |
| Repository | |
| Written in | Python |
| Operating system | Microsoft Windows, macOS, Linux |
| Available in | Python |
| Type | Workflow management platform |
| License | Apache License 2.0 |
| Website | airflow.apache.org |
Apache Airflow is an open-sourceworkflow management platform. It started at Airbnb in October 2014[2] as a solution to manage the company's increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface.[3][4] From the beginning, the project was made open source, becoming an Apache Incubator project in March 2016 and a Top-Level Apache Software Foundation project in January 2019.
Airflow is written in Python, and workflows are created via Python scripts. Airflow is designed under the principle of 'configuration as code'. While other 'configuration as code' workflow platforms exist using markup languages like XML, using Python allows developers to import libraries and classes to help them create their workflows.
Overview[edit]
Airflow uses directed acyclic graphs (DAGs) to manage workflow orchestration. Tasks and dependencies are defined in Python and then Airflow manages the scheduling and execution. DAGs can be run either on a defined schedule (e.g. hourly or daily) or based on external event triggers (e.g. a file appearing in Hive[5]). Previous DAG-based schedulers like Oozie and Azkaban tended to rely on multiple configuration files and file system trees to create a DAG, whereas in Airflow, DAGs can often be written in one Python file.[6]
Managed Providers[edit]
Three notable providers offer ancillary services around the core open source project. Astronomer has built a SaaS tool and Kubernetes-deployable Airflow stack that assists with monitoring, alerting, devops, and cluster management.[7] Cloud Composer is a managed version of Airflow that runs on Google Cloud Platform (GCP) and integrates well with other GCP services.[8] Starting from November 2020, Amazon Web Services offers Managed Workflows for Apache Airflow.[9]
References[edit]
Airflow Case
- ^'Announcements - Apache Airflow'. airflow.apache.org. The Apache Software Foundation. Retrieved 2021-03-16.
- ^'Apache Airflow'. Apache Airflow. Archived from the original on August 12, 2019. Retrieved September 30, 2019.
- ^Beauchemin, Maxime (June 2, 2015). 'Airflow: a workflow management platform'. Medium. Archived from the original on August 13, 2019. Retrieved September 30, 2019.
- ^'Airflow'. Archived from the original on July 6, 2019. Retrieved September 30, 2019.
- ^Trencseni, Marton (January 16, 2016). 'Airflow review'. BytePawn. Archived from the original on February 28, 2019. Retrieved October 1, 2019.
- ^'AirflowProposal'. Apache Software Foundation. March 28, 2019. Retrieved October 1, 2019.
- ^Lipp, Cassie (July 13, 2018). 'Astronomer is Now the Apache Airflow Company'. americaninno. Retrieved September 18, 2019.
- ^'Google launches Cloud Composer, a new workflow automation tool for developers'. TechCrunch. Retrieved 2019-09-18.
- ^'Introducing Amazon Managed Workflows for Apache Airflow (MWAA)'. Amazon Web Services. 2020-11-24. Retrieved 2020-12-17.
External links[edit]
Airflow Open Source