Migrating to Durable Functions

One of the things I love about Azure Durable Functions is how easy it makes it to implement tricky workflow patterns like fan out and fan in (running workflow steps in parallel), and waiting for notifications from an external system with timeout support.

And in typical enterprise applications there are lots of workflows that would benefit from being written as Durable Orchestrations. The trouble is, it's not always a trivial task to take an existing complex workflow and rewrite the whole thing in Durable Functions.

Fortunately, that's not necessary. It's possible to simply use Durable Functions for the orchestration part of the code, and leave the implementation of the individual steps in the workflow where they were.

And you don't even necessarily need to convert the entire workflow in one hit. It can be done incrementally, with a Durable Functions orchestration managing part of the workflow and then handing back over to the legacy implementation. You might just want to start off by moving the complex stuff like fan out fan in into a Durable Functions orchestration initially.

How does this work?

A Durable Functions orchestrator function defines the steps (or "activities") in a workflow - starting with what to do first, and then what should happen next when each individual step completes. The orchestrator doesn't perform any of the actual activities itself. Those are normally implemented as "activity functions".

However, those activity functions can simply trigger behaviour implemented elsewhere. This might be by making a HTTP request, or posting a message to a queue. So it's pretty straightforward to trigger your legacy workflow step implementations.

Waiting for those steps to finish is a little bit trickier. How does the legacy code inform our orchestrator that it has completed the step and report back the result? There are a few different options.

Option 1 - Polling

Your external workflow step implementation might support polling for completion. That can be a bit cumbersome to write in Durable Functions, since it's not a good practice for an Activity Function to sleep (since they are billed by the second and are intended to be relatively short-running).

This means that after kicking off the potentially long-running action, you'd need to return to your orchestrator, which then sleeps for a bit (with CreateTimer). When it wakes up it call another activity function which makes the HTTP request to poll for progress. That activity function then tells the orchestrator whether the operation has completed or not.

However, thanks to the Durable HTTP requests feature I wrote about recently, that whole process is simpler, so long as there is a suitable endpoint that can be polled for progress returning 202 until the operation has completed. This API isn't currently quite as flexible as I'd like, but if you can use it, it will greatly simplify your overall orchestrator code.

Option 2 - External Events

The second option is for the workflow step to notify you when it's completed. This could be by calling a web-hook, or posting a message onto a queue. Then all you need is a regular Azure function that receives that notification, and passes it on to your durable orchestration by means of RaiseEventAsync. The orchestrator then simply needs to call WaitForExternalEvent

Of course the external activity could post the event directly to the target orchestration because Durable Functions exposes an API that you can call to trigger external events. However, I prefer to leave a level of abstraction in between, as I don't think that the code that implements a step in a workflow should be tightly coupled to the specific technology that orchestrates it.

Migrating slowly

With these two approaches to triggering activities implemented outside your Durable Function app, it is possible to incrementally migrate large and complex workflows into Durable Functions. By doing so you'll end up with multiple benefits of using Durable Functions.

Of course, if you are planning to slowly evolve these workflows over time, you do need to make sure you've thought about a good versioning strategy for your orchestrators, as changing an orchestrator function can cause in-progress orchestrations to fail.