Reactive Durable Functions

Durable Functions itself are a big topic and I'll come back to the them soon. In the previous post I created a simple Function App, which inserts a row into Table Storage. All activities were orchestrated and it was really easy to schedule more work. Today I want to present you how easy you can transform such active architecture into a passive one using Event Grid. This will be a fairly easy episode so let's start!

Function App

There's no need to change anything in the Function App since for now we'll use a HttpTrigger. Unfortunately we cannot host it locally(because of Event Grid) so there's a need to publish it to the cloud.

Event Grid

You can easily deploy Event Grid from the marketplace. For now there's nothing special regarding its installation, so I won't go into details.

Event Grid chosen from Marketplace - still in preview though

Combining it all together

Once we have components deployed we can configure Event Grid so it'll pass events to the chosen endpoint. In our case it'll be our Galaxy_Create_Start function(which we used to start orchestration). We'll need one thing - our function URL. To get it go to the Function App you deployed, find a function and click on Get Function URL.

Once you have it, we can go to Event Grid and create a new subscriber.

What we need now is a new subscription. This feature allow you to orchestrate events flow, so each subscriber can be subscribed to a particular event type. With this configured we can centralize the way, how e.g. multiple services built with Azure Functions integrate with event producers.

My subscriber configured and subscribed to nebula.galaxy event type

When you configure a new subscriber it will be added to the list of all supported subscribers. We've done all what we needed to integrate Event Grid with Azure Functions, let's test it now.

Working example

There're two important things what we need to test our solution - Event Grid endpoint and access key. The former ss available on the main screen - please copy it so you'll know where to post your messages. Access keys can be found under Settings section in Event Grid main menu.

Since we don't have any real producer yet, we'll try to simulate one. For this purposed I used Postman, however all is up to you. The only thing we have to do is to post a HTTP request to the mentioned enpoint. Here you have an example:

/
POST /api/events HTTP/1.1
Host: your.eventgrid.azure.net
aeg-sas-key: YOUR_KEY
Content-Type: application/json
Cache-Control: no-cache

[
    {
        "id": "2",
        "eventType": "nebula.galaxy",
        "subject": "nebula/galaxy/create",
        "eventTime": "2017-11-08T13:25:00+01:00",
        "data":{
        }
    }
]

As you can see the payload has a specific schema, which will be validated on the Event Grid side. In fact it should self-explanatory. What is important here is the fact, that this payload is being passed to the function. If you change the main function a little bit:

/
[FunctionName("Galaxy_Create_Start")]
public static async Task<HttpResponseMessage> StartOrchestration(
	[HttpTrigger(AuthorizationLevel.Function, "post", Route = "orchestration/start")] HttpRequestMessage req,
	[OrchestrationClient] DurableOrchestrationClient starter,
	TraceWriter log)
{
	// Function input comes from the request content.
	string instanceId = await starter.StartNewAsync("Galaxy_Create", null);

	var payload = await req.Content.ReadAsStringAsync();
	log.Info($"Started orchestration with ID = '{instanceId}'.");
	log.Info($"The payload is: {payload}");

	return starter.CreateCheckStatusResponse(req, instanceId);
}

You'll see following result when you go to the function and check the console:

/
2017-11-08T12:36:54.565 Function started (Id=ecb2655e-912e-435b-b916-f21b65729716)
2017-11-08T12:36:55.144 Started orchestration with ID = 'f7d2ad0001204ff0a381d61b448ef8b7'.
2017-11-08T12:36:55.144 The payload is: [{
  "id": "3",
  "eventType": "nebula.galaxy",
  "subject": "nebula/galaxy/create",
  "eventTime": "2017-11-08T12:25:00+00:00",
  "data": {},
  "topic": "/SUBSCRIPTIONS/____________/RESOURCEGROUPS/NEBULA-EUW-DEV-RG2/PROVIDERS/MICROSOFT.EVENTGRID/TOPICS/NEBULA-EUW-DEV-EVENTGRID"
}]
2017-11-08T12:36:55.173 Function completed (Success, Id=ecb2655e-912e-435b-b916-f21b65729716, Duration=596ms)

Of course you can easily deserialize it and incorporate into your flow.

Summary

As you can see implementing Event Grid as a gateway to the underlying architecture is a piece of cake. In fact we didn't need any change in our code - the whole integration perfomed seamlessly. In the next episode I'll try to present you how to integrate a producer(Event Hub), so we don't have to post messages directly to our Event Grid.

What's all about Azure Event Grid?

Azure Event Grid is one of the newest products available in Azure cloud stack. Since it's still in preview, we are not offered full functionality(so e.g. only two regions can be selected, not all event publishers have been added). However with all the goodnes provided by this component, we can start thinking about "reactive programming in the cloud" - at least this is what documentation tells us. Let's dive deeper into Event Grid and find why it's so special.

Competitors

Event Grid is all about events. You may ask how it is different comparing similar products like Event Hub or Service Bus. If you take a look at the basic architecture, you'll find very similar concept like topics or subscribers (well at least for Service Bus). So why do I need Event Grid(which will complicate my architecture even more) when I can easily connect e.g. my Azure Functions to a topic and achieve the same functionality with ease? Well, this is only partially true.

Event Grid functional model(source: https://docs.microsoft.com/en-us/azure/event-grid/overview)

The downside of other solutions is the need of pooling - details doesn't matter now, you have to implement some way of communication between your app and an event publisher. It can be long-pooling, event sourcing, WebSockets - whatever works can be used. So even if you establish a persistent connection, you have to talk to the other side and await messages. You're not passive in this model - that's why you cannot "react" on events passed to you. You only parse them and pass further.

Event Grid allows you to make your components "passive" - they are somewhere in the cloud and are only interested in the data you send to them. They don't have to persist any connections - it's up to Event Grid to distribute messages and deliver to the configured subscribers. Microsoft states, that this approach is suited for serverless scenario and I can agree with them - you can make underlying infrastructure even more abstract and control the flow of event from the single point. For me the possibility to configure connection between Event Hub and several Azure Functions using Azure Portal(so I don't have to pass a connection string of EH to each individual component) is definitely a big YES to Event Grid.

Should I go for it?

I still think, that though Event Grid simplifies and improves working with serverless architecture(what am I saying - it actually enables you to start thinking about serverless at all...), you cannot just take it, write a couple of Functions and say "this is how we're making applications today in our company". It still requires proper planning, it's still not valid for each and every application(with Event Hub, Event Grid and Azure Functions, you may assume, that an event will reach its destination... at some point in time) and forces you to change your mindset into being "reactive"(and this is sometimes a challenge itself).

Event Grid as the "man-in-the-middle" in serverless architecture(source: https://docs.microsoft.com/en-us/azure/event-grid/overview)

On the other hand I like how it smoothly integrates with the cloud - for now only a few publishers are available, but we're given a promise, that this will change soon. I treat it as a serverless orchestrator - it's the centre of my architecture, where I can separate concerns seamlessly. Combine it with negligible cost($0.60 per million operations, first 100k is free) and easy learning curve and ask yourself why haven't you tested it yet?