Combining Azure Service Fabric and Service Bus

Combining Azure Service Fabric and Service Bus

To integrate Service Fabric into an existing application landscape, it often makes sense to introduce a queuing mechanism.

For example, it could make sense to have a back-office shipping system notify services running in Service Fabric about completed shipments.AzureServiceFabricAndServiceBus

Using a reliable queue effectively decouples the two systems. It allows for downtime in either system, without negatively affecting the other. Obviously, the queue system itself needs to be rock solid, fortunately such systems are readily available. One option is to use Azure Service Bus.

You can find all sources here.

Receiving messages

To receive messages inside a Service Fabric service, you can use QueueClient from the package `Microsoft.Azure.ServiceBus`. By combining the service bus queue client with the Service Fabric communication mechanism defined by `ICommunicationListener`, you can create a solution that is reusable within all your services. This approach will connect the Service Bus client’s life cycle to the Service life cycle. You don’t need to worry about opening and closing the connection, as you would when using the Service Bus client directly within your service. You can focus on the handling of messages instead.

The implementation of the OpenAsync would look similar to this:

/// <summary>
/// This method causes the communication listener to be opened. Once the Open
/// completes, the communication listener becomes usable - accepts and sends messages.
/// </summary>
/// <param name="cancellationToken">Cancellation token</param>
/// <returns>
/// A <see cref="T:System.Threading.Tasks.Task">Task</see> that represents outstanding operation. The result of the Task is
/// the endpoint string.
/// </returns>
public override Task<string> OpenAsync(CancellationToken cancellationToken)
    var builder = new ServiceBusConnectionStringBuilder(ReceiveConnectionString);
    if (string.IsNullOrWhiteSpace(builder.EntityPath))
        builder.EntityPath = QueueName;
    ServiceBusClient = new QueueClient(builder, ReceiveMode, RetryPolicy);
    if (MessagePrefetchCount > 0)
       ServiceBusClient.PrefetchCount = MessagePrefetchCount;
    string uri = SendConnectionString;
    return Task.FromResult(uri);

A new queue client is created, with some settings that were provided to the constructor. It will be configured with a method that is invoked for every received message.

ServiceBusClient.RegisterMessageHandler(ReceiveMessageAsync, options);

The callback method needs to inform the queue client whether it has processed the message or not. In the first case, the message must be removed from the queue (‘completed’).  In the second case, the message needs to be either moved to the dead-letter queue or put back on the queue to be retried later. A clean way to implement this is by passing the message to a separate handler. Every service using the component would be able to provide its own handler. We’ll name it `DefaultServiceBusMessageReceiver`. Types inheriting from this abstract implementation will have the following operations at their disposal:

  1. Receive – will be called every time a message arrives
  2. Complete – can be used to complete a message
  3. Abandon – to abandon the lock on a message, so it will be redelivered later
  4. Dead-letter – to move a message to the dead-letter queue

The calls will be forwarded to the communication client which in turn will call the queue client.

When the receiving service shuts down, the queue client needs to be closed, so it won’t receive any more messages.

/// <summary>
/// This method causes the communication listener to close. Close is a terminal state and 
/// this method allows the communication listener to transition to this state in a
/// graceful manner.
/// </summary>
/// <param name="cancellationToken">Cancellation token</param>
/// <returns>
/// A <see cref="T:System.Threading.Tasks.Task">Task</see> that represents outstanding operation.
/// </returns>
protected override async Task CloseImplAsync(CancellationToken cancellationToken)
      await ServiceBusClient.CloseAsync();

That’s it!

Visit my GitHub to see all the code or simply install the Nuget package.

Note that there is a version that supports the Full .NET Framework as well. This has support for batching and sessions, which is not yet ported to the Netstandard version.

Author: loekd

Loek is a Technical Trainer, Cloud solution architect at Xpirit, a public speaker and Microsoft Azure MVP. He focuses on creating secure, scalable and maintainable systems. To help companies make the most efficient transition into the Cloud, he is always looking for even better ways to leverage the Microsoft Azure Platform. As an active member of open source projects, Loek likes to exchange knowledge with other community members. Let’s engage!

8 thoughts on “Combining Azure Service Fabric and Service Bus”

  1. Hey, very interesting article, how would you do it for a .net core project as the CreateServiceInstanceListeners function is quick different.


    1. Thanks!
      You can use the netstandard package in your .net core projects. It’s currently in preview.


      Use this code to create a listener:

      var listener = new ServiceBusQueueCommunicationListener(cl => new DefaultServiceBusMessageReceiver(cl, null), serviceContext, serviceBusQueueName, serviceBusSendConnectionString, serviceBusReceiveConnectionString);


      1. Sorry to bother you again but I just want to make sure that I understand it correctly.

        Create a Receiver which inherits from DefaultServiceBusMessageReceiver
        Register listeners

        For sending messages I can just create a QueueClient, right?
        Is there ever a difference between serviceBusSendConnectionString and serviceBusReceiveConnectionString?


  2. Yes, to the first two questions.

    About the connection strings: they should ideally be different, so a client can only send messages and a receiver can only receive them. This best practice is known as ‘least privilege’.

    Make sure to generate separate read & write ‘shared access policies’ for every individual Queue and Topic, don’t just use RootManageSharedAccessKey because it has way too many privileges.

    Also note that in this library, the serviceBusSendConnectionString is only needed if you also use the client package.


  3. Hi Loekd, Great work so far!

    I first stumbled upon your ServiceFabric PubSub library, then when we started to have porblems with the scaling looked into alternatives like ServiceBus and again stumbled upon a library from you, the one linked in this article.

    I am curious why you created both and what were the intended use cases for both approaches, could you please provide some insights? vs

    We require strict ordering between different event types per topic.
    We would like to be able to archive the events in some way for auditing, debugging etc.
    At least once delivery should be guaranteed, while exactly once would be a dream.
    Scaling is an issue as we need to support multiple production sites which all produce events.

    Considering this what would you suggest? Service Bus or a custom Broker or something totally different?


    1. Hi, thanks for your comments. The use cases for both solutions are different. If you want to create an event-driven architecture within ASF without introducing external dependencies, you’d use the PubSub library. If you need to communicate to parties outside the cluster, or if you don’t mind having the external dependency to ASB, you’d use the ServiceFabric.ServiceBus library.

      The PubSub library keeps message ordering in tact, provided you use the standard setup, not the more performant but unordered alternative.

      Archiving events / messages would mean either adding an additional ‘archiving’ subscriber to every publisher, or adding some custom code to the broker service.

      In the PubSub lib, every subscriber gets its own inbox, so delivery should be at least once. Similar to using ASB.

      If your information needs to leave the cluster at some point, or when you have a great volume of messages to send around, you can’t use the PubSub lib, and should use ASB (or something similar).


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s