Messaging with Azure Service Bus - Part 1 - Why Use Messaging?

In this series:

• Part 1 - Why Use Messaging?
• Part 2 - The Challenges of Messaging
• Part 3 - Introducing Azure Service Bus
• Part 4 - Sending and Receiving Messages
• Part 5 - Message Sending Options
• Part 6 - Message Receiving Options
• Part 7 - Topics and Subscriptions
• Part 8 - Commands and Events
• Part 9 - Premium Features

Messaging is an extremely useful and powerful architectural technique that provides many benefits especially in cloud-based applications. This article is the first in an introductory series in which I want to explore how Azure Service Bus can be used to implement all kinds of useful messaging patterns.

In this first installment, I want to talk about why messaging is so useful. Later in this series, we'll explore some of the challenges that messaging can bring, as well as seeing lots of simple examples of how to implement some of the core messaging patterns with Azure Service Bus. I also want to demonstrate how to take advantage of some of its more advanced features.

What is messaging?

The basic idea of messaging is quite a simple one. If I want to ask you a question, I could phone you, and ask you directly. But maybe it's not a convenient time for you to talk. So as an alternative, I could send you a text message. That way, you can respond at a time that is convenient to you, although I might have to wait a bit longer to get my response. That's why sending messages is often referred to as asynchronous communications, as it doesn't require both parties to be online at the same time.

In software applications, if I want to send a message, I simply take the data I want to send, and serialize it (JSON is a common choice. The message is sent to what is often called a "message bus" or "message broker". This acts as an intermediate party, which holds onto the message until it can be delivered to the intended recipient. The recipients of the messages will often "poll" the message bus, asking if there are any messages available. This means that messages can be sent, even if the receiving program isn't currently running.

Why use messaging?

There is no requirement that you use messaging. In fact, I managed to go quite a long period through my career without ever using messaging at all. Most applications I created had simple web or Windows-based user interfaces, and when the user clicked on a button, all the actions that needed to take place in response to that user action were done there and then. That approach has its benefits - it's pretty straightforward for one thing, and it also makes it easy to report back to the user interface if anything has gone wrong.

But messaging brings several very valuable benefits. One is that it is a very effective way of decoupling the sender of the message from the recipient. In other words, so long as I can send a message, I don't need to care about who is handling that message. It could be a completely different application, written in another language, by another development team. The recipient just needs to know how to receive the message.

Messaging allows code to appear more responsive. Suppose you place an order in an online shop. There might be many things that need to happen after the order is placed. But you as the user of the website don't need to wait for all that to be done. All you need to know is that your order has been accepted. And a message is a great way of triggering the additional work that needs to be done, without forcing the user to wait for it to happen.

Messaging also opens the door to scaling out for performance reasons. Suppose I have an application that suddenly gets a huge burst of work to do. It might easily overwhelm the compute resources I have available. But if I can package that work into individual tasks, each represented by a message, then all I need to do is post those messages to the message bus. Those messages essentially end up in a queue, which a worker slowly works through, handling each message. But this architecture also allows for the possibility of multiple workers to work in parallel, allowing us to much more quickly work through the messages. Some platforms like Azure Functions do this scaling out automatically.

Messaging also is helpful for implementing retries. If a message cannot be processed the first time around, many messaging systems including Azure Service Bus allow for that message to be retried a certain number of times. If the message still can't be processed it can even be moved to a special location called a "dead letter queue" allowing for some other process to attempt to recover from the error. Features like this mean that message based systems can actually be very resilient to all kinds of transient errors.

There are many other benefits to the use of messaging. For example, messages are extremely useful when you're trying to build "workflows" where a complex business process goes through many stages. Azure Durable Functions is a great example of a platform that allows you to build reliable workflows, and under the hood messaging is what is powering the whole system.

I'll leave this here for now, and in the next installment I want to talk about some of the challenges of messaging.