Airflow 2.7 Is Now Out

Here are the most important feature updates that will make your life easier and save you time

Image generated via DALL-E-2 using prompt “data flowing from external system into destination system as a system design graph, digital art”

Apache Airflow 2.7.0 is finally out and we are all excited to see all these notable features being shipped in this latest release. The new version consists of 40 new features, 53 bug fixes, 49 improvements and 15 documentation updates.

The main focus of this release has been security but at the same time, many exciting non-security related features have also been made available.

The new Cluster Activity UI

As of Airflow 2.7.0, a new tab is introduced in the top level menu of the Airflow UI and is called Cluster Activity.

The new Cluster Activity UI gives an overview of the overall cluster state, including component health (for MetaDatabase, Scheduler, Triggerer and DAG processor) as well as details about DAG/Task run states and DAG run types.

The new Cluster Activity UI is shipped as part of the new Airflow 2.7.0 release — Source: Author

See when the source code was last parsed

In the past I was really annoyed by the fact that I couldn’t really see whether the changes I was making to the source code of a particular DAG were actually parsed. I would usually have to refresh the page (multiple times) and start looking for the specific areas in the source code that I applied some changes in order to ensure that they have been parsed so I can re-trigger the DAG.

The new Airflow version introduces a Parsed at field within the Code tab of a DAG, indicating the timestamp when the DAG’s source code was last updated.

The new code tab includes a timestamp indicating when the DAG source code was last parsed — Source: Author

Simple but yet useful addition!

Keyboard shortcut support

Airflow Grid view now also supports keyboard shortcuts. Once you enter the Grid screen of a DAG, you will notice a note just underneath filtering section, indicating that you can access the list of supported shortcuts…

Source link

Leave a Comment