Using Azure Service Bus Filtered Subscriptions

Azure Service Bus Topics and Subscriptions offer you a powerful way to configure multiple subscribers for a single message. So you could post an OrderCreated message to a topic, and one subscriber might initiate payment processing, another might send a confirmation email, and another might write to an audit log. So using topics and subscriptions instead of queues offers you a lot of flexibility.

But sometimes in a topics and subscriptions setup you might find your subscriber is only interested in receiving a subset of the messages posted to that topic. One way to handle this is for your subscribers to simply ignore the messages they don’t care about, but that is wasteful since you pay for every receive operation.

A better approach is to create a filtered subscription which will only receive messages you are interested in. A filter can be based on any properties of the BrokeredMessage, except the body, since that would require every message to be deserialized in order to be handed on to the correct subscriptions.

Let’s see how easy it is to get started with filtered subscriptions.

Step 1 - Create Your Topic

Subscriptions are based off of topics, so we need to ensure we have a topic. Here’s some simple code to create a topic if it doesn’t already exist:

string connectionString = // your servicebus connection string here;

// the names of topics and subscriptions we'll be working with
const string topicName = "MyTestTopic";
const string allMessagesSubName = "AllMessages";
const string filteredSubName1 = "Filtered1";
const string filteredSubName2 = "Filtered2";

// let's create the topic if it doesn't already exist...
var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.TopicExists(topicName))
{
    var td = new TopicDescription(topicName);  
    namespaceManager.CreateTopic(td);
}

Step 2 – Create the Filtered Subscriptions

You can create subscriptions at any time, but one important thing to remember with topics is that unlike queues, if no one is listening, messages sent to the topic will be lost. So make sure you create your subscriptions before sending messages to that topic.

Let’s create three subscriptions on our topic. The first one will have no filter, and so will receive all messages.

The second is going to filter on a user defined message property called “From” that we will add to each message we send. It uses a SqlFilter which lets us use a SQL like syntax to say that we want all messages whose ‘From’ property ends in ‘Smith’.

Our third subscription will use a built-in property of the BrokeredMessage. We want to only receive messages whose Label property is set to “important”. We have to prefix Label with sys. in order to indicate this is a built-in property rather than one of the custom user defined properties.

if (!namespaceManager.SubscriptionExists(topicName, allMessagesSubName))
{
    namespaceManager.CreateSubscription(topicName, allMessagesSubName);
}

if (!namespaceManager.SubscriptionExists(topicName, filteredSubName1))
{
    namespaceManager.CreateSubscription(topicName, filteredSubName1, new SqlFilter("From LIKE '%Smith'"));
}

if (!namespaceManager.SubscriptionExists(topicName, filteredSubName2))
{
    namespaceManager.CreateSubscription(topicName, filteredSubName2, new SqlFilter("sys.Label='important'"));
}

Step 3 – Send Some Messages

You don’t send your messages directly to a subscription, you send them to the topic, and that will forward them to all the relevant subscriptions based on their filters.

Here we send three messages, setting up the “From” user defined property for each one, and also setting the built-in Label property for two of them.

var message1 = new BrokeredMessage("Hello World");
message1.Properties["From"] = "Ian Wright";

var message2 = new BrokeredMessage("Second message");
message2.Properties["From"] = "Alan Smith";
message2.Label = "important";

var message3 = new BrokeredMessage("Third message");
message3.Properties["From"] = "Kelly Smith";
message3.Label = "information";

var client = TopicClient.CreateFromConnectionString(connectionString, topicName);
client.Send(message1);
client.Send(message2);
client.Send(message3);
client.Close();

Step 4 – Receive Messages

Now we need to listen on each of those three subscriptions. I’m just going to use the SubscriptionClient’s ReceiveBatch method to pull off a batch of messages from each subscription. Here’s how to perform ReceiveBatch for a single subscription:

var subClient = SubscriptionClient.CreateFromConnectionString(connectionString, topicName, subscriptionName);

var received = subClient.ReceiveBatch(10, TimeSpan.FromSeconds(5));
foreach (var message in received)
{
    Console.WriteLine("{0} '{1}' Label: '{2}' From: '{3}'", 
        subscriptionName,
        message.GetBody<string>(),
        message.Label,
        message.Properties["From"]);
}

subClient.Close();

If this works correctly, our unfiltered subscription will receive all three messages, the "From LIKE '%Smith'" filter will get two, and the "sys.Label='important'" filter will get one:

AllMessages 'Hello World' Label: '' From: 'Ian Wright'
AllMessages 'Second message' Label: 'important' From: 'Alan Smith'
AllMessages 'Third message' Label: 'information' From: 'Kelly Smith'
Filtered1 'Second message' Label: 'important' From: 'Alan Smith'
Filtered1 'Third message' Label: 'information' From: 'Kelly Smith'
Filtered2 'Second message' Label: 'important' From: 'Alan Smith'

Perfect!

Bonus Step – Modifying Your Filters

You might get into a situation where you’ve already created a subscription but now you want it to be filtered. It is possible to change the filter, by deleting the default “rule” that was created when you initially created the subscription (which will be called “$Default”), and creating your own new rule with the new subscription. For safety, let’s add the new rule before we delete the old one, to eliminate the chance that we miss a message we wanted during the window when there are no rules.

var subClient = SubscriptionClient.CreateFromConnectionString(connectionString, topicName, subscriptionName);

var newRule = new RuleDescription("FilteredRule", new SqlFilter("From LIKE '%Smith'")); 

subClient.AddRule(newRule);

subClient.RemoveRule("$Default");

And now your subscription has the new filter applied. Note that any messages already forwarded to this subscription before this filter was applied will still be in the subscription even if they don’t match the new filter.