#AIRFLOW 2.0 UPGRADE#

A copy of the configuration file also helps during the upgrade to take all settings with you. Nevertheless, you should take the following tips to heart:Ĭreate a backup of the used database to avoid losing your historical workflow runs. The necessary steps and background information are summarized there. Practical tips for the changeoverįor a successful upgrade, you can refer to the detailed documentation provided by Airflow. For users, the separation manifests itself in the customization of operator paths in new and old workflows. The provider packages are also detached from the official Airflow releases and bug fixes are implemented faster. If Airflow is loaded locally on a single server, the Kubernetes package can be left out. This measure is intended to keep the installation more lightweight.

This is also the focus in the further development of Apache Airflow.While initially only the design was adapted, in the future Airflow will be transformed into a modern web application with an appealing technology stack.Īlso relevant for users is the separation of Apache Airflow into a main application and provider packages. What is new, however, is the revised, modern web user interface. Typical use cases remain and Apache Airflow best practices are still valid. Accordingly, it is still possible to start a workflow via the web interface and the workflows are still available in a Python file. Some aspects remain the same in the new major version of Airflow. Here, the last working day of the month will be a valid selection in the future. For example, in the current development pipeline is a more extensive possibility to define the execution times of a workflow. Of course, further functionality will be added as part of the lifecycle of the new major release. In the calendar view, you can see the success rate of your workflows broken down graphically by day of the week and month.Īs soon as you want to follow a workflow live, the auto-refresh functionality comes in handy: The status of the tasks are automatically updated in the graph view and the page does not have to be reloaded. With two new functions, you can keep a perfect overview of your workflows. Access authorization can easily be done via user interface authentication.

Variables, connections, workflows and users are conveniently controlled via an HTTP request. While the old API remains intact for the time being, a new REST API based on the OpenAPI specification offers new possibilities for automation and integration scenarios between used systems.

The introduced optimizations lead to a convincing speed advantage even in the case of a single scheduler. For larger installations, multiple scheduler instances can now be used, working together in an active-active design. Since the major release, the scheduler component has undergone a major overhaul. What does this mean in concrete terms in version 2.1? The new features thus promise user-centric workflow management at the pulse of current needs. As an open-source tool, Apache Airflow is further developed by a wide-ranging community, which develops feature ideas from its own use cases. To increase the motivation to upgrade, it's worth taking a look at the new features. To ensure that nothing stands in the way of a problem-free changeover, in this articel we’ll provide you with practical tips on what to bear in mind during the upgrade. Nevertheless, upgrading involves effort and carries the risk of data loss and temporary system downtime. Moving to the new major release brings new features, faster workflow execution, and improved security. If you haven't considered upgrading before, now is a good time to do so. The new major release of the Apache Airflow workflow management platform will soon be one year old.

Airflow 2.0