Recently Ive been looking at Apache Airflow since Ive noticed it getting a lot of attention from Python developers and cloud providers for supporting workflow scenarios. For context, Im the creator of Durable Functions on Azure and well soon be announcing the general availability (GA) of Durable Functions for Python. Both Airflow and Durable Functions support building workflows in Python, so I thought it would be worth doing some investigation to understand the differences between the two technologies. This blog post is my attempt at doing this comparison and I hope folks find it somewhat useful. In the end, what Ive learned is that Durable Functions and Apache Airflow are trying to solve different problems using different approaches, in spite of the fact that they both support implementing workflows in Python. The main differences are around the types of workflows supported and the role that Python plays in the authoring process. I go into more details on these differences in this post. Orchestrators vs DAGs The most important technical difference Ive found is the programming model. Durable Functions allows you to build workflows by writing orchestrator functions. Orchestrator functions describe how actions are executed and the order in which actions are executed. To illustrate, the following is a simple sequential orchestration in Python that calls three tasks known as activities, t1, t2, and t3, in sequence using Pythons generator syntax. Each yield expression causes the orchestrator function to wait for the scheduled task to complete and then save the result to a local variable for later use. When yielded in this way, the orchestrator function can be unloaded from memory and the progress of the workflow is persisted. An orchestration can have many different types of actions, including activity functions, sub-orchestrations, they can wait for external events, make HTTP calls, and sleep using durable timers. Orchestrator functions can also interact with durable actor-like objects known as entity functions. All of this is done using normal procedural coding constructs, which means you can use programming language features like conditionals, loops, function calls, exception handling via try/except/finally (for implementing compens
