The sample demonstrates how Windows Azure developers can leverage the Windows Azure Service Bus to enable asynchronous inter-role communication in cloud-based solutions using Service Bus topics and subscriptions. The code sample offer a complete solution, one that enables simplified, scalable, loosely coupled communication between role instances on the Windows Azure platform.

Step-By-Step Implementation Guide

Below is a step-by-step instruction on how to enable a Windows Azure solution to support the inter-role communication mechanism discussed in this post.

Step 1: Create a Service Bus Namespace

You will first need to provision a Service Bus namespace. If you already have an existing namespace, it can be reused for inter-role communication. In either case, you will need to note down the three key elements: namespace name, issuer name and issuer secret.

7-12-2011 3-53-17 PM111

The three highlighted parameters will be required for configuring the inter-role communication component in step 3.

Step 2: Add an Inter-Role Communication Component to a Window Azure Solution

In this step, you will add a reference to Microsoft.AzureCAT.Samples.InterRoleCommunication.dll assembly to every project implementing either web or worker roles in a Visual Studio solution. In addition, add a reference to Microsoft.ServiceBus.dll from the AppFabric SDK. The location of this assembly depends on the version of the SDK installed on your machine. For example, the Windows Azure AppFabric SDK v1.5 resides in %ProgramFiles%\Windows Azure AppFabric SDK\V1.5\Assemblies\NET4.0\.

ImpNoteNoBgImportant
It is very important to set the Copy Local property of the respective assembly references to True. Otherwise, the role instances may not initialize and spin themselves off successfully when deployed on Windows Azure.

Step 3:  Configure the Inter-Role Communication Component

Once a reference to the assembly containing the IRC component has been added to the respective projects, the next step is to provide some configuration information to enable the IRC component to connect to the target Service Bus topic. Such a configuration will need to be supplied in the app.config file in each Visual Studio project implementing a web or worker role. The respective app.config file is to be modified as shown:

<configuration>
  <configSections>
    <!-- Add this line into your own app.config -->
    <section name="ServiceBusConfiguration" type="Microsoft.AzureCAT.Samples.InterRoleCommunication.Framework.Configuration.ServiceBusConfigurationSettings, Microsoft.AzureCAT.Samples.InterRoleCommunication" />
  </configSections>

  <!-- Add this entire section into your own app.config -->
  <ServiceBusConfiguration defaultEndpoint="IRC">
    <add name="IRC" serviceNamespace="irc-test" endpointType="Topic" topicName="InterRoleCommunication" issuerName="owner" issuerSecret="wjc/h98iL95XudZPJ1txaDB..." />
  </ServiceBusConfiguration>
</configuration>

In your version of app.config file, 3 parameters in the above fragment will have to be modified:

  1. Service Bus namespace (see step 1).
  2. Issuer name (see step 1).
  3. Issuer secret (see step 1)

If required, you can also modify the name of the Service Bus topic that will be used to exchange the IRC events.

Double Quote  Note
If you have multiple Visual Studio projects implementing web/worker roles which need to participate in the inter-role communication, you will need to ensure that the app.config file is present in each project. Instead of creating multiple copies of app.config and placing them in each individual project, you can add app.config as a link rather than directly adding the file to your projects. Linked project items in Solution Explorer can be identified by the link indicator in its icon (a small arrow in the lower left corner). By linking to a file, you can capture ongoing changes to the configuration file without having to manually update a copy whenever changes are made. To create a link to a file, select the target project, open the Add Existing Item dialog box, locate and select the master app.config file you want to link. From the Open button drop-down list, select Add As Link.

In addition to configuration settings expressed inside an app.config file, the IRC component supports several useful runtime settings that are intended to be programmatically modified if their default values are not sufficient. These runtime settings are combined in the InterRoleCommunicationSettings object and are listed below:

Setting Description Default Value
EventTimeToLive Specifies a custom time-to-live (TTL) value for inter-role communication events. Events older than their TTL will be removed before they can reach the subscribers. 10 minutes
EventWaitTimeout Specifies a custom time value indicating how long Service Bus messaging API will keep a communication channel open while waiting for a new event. 30 seconds
EventLockDuration Specifies a custom duration in which the inter-role communication events that are being processed remain marked as “in-flight” and invisible to other competing consumers (if any). 5 minutes
EnableAsyncPublish Enables publishing the inter-role communication events in an asynchronous, non-blocking fashion eliminating the need to wait until events are put into a topic. False
EnableAsyncDispatch Enables dispatching the inter-role communication events to their subscribers immediately as soon as a new event arrives, without waiting until the previous event is processed. True
EnableParallelDispatch Enables dispatching the inter-role communication events to multiple subscribers in parallel without serializing execution of subscribers. True
EnableCarbonCopy Enables the publishing role instances to receive their own events as carbon copies. False
UseCompetingConsumers Enables multiple role instances to receive the inter-role communication events from the same subscription with the guarantee that each event is consumed only by one of the role instances. False
RetryPolicy Specifies a retry policy that will be used for ensuring reliable access to the underlying messaging infrastructure. Fixed Interval

Step 4: Initialize the Inter-Role Communication Component

In this step, you will extend the startup logic of your web or worker role implementation with a code snippet that is responsible for initializing the IRC component. The following example demonstrates how to initialize the IRC component from within a worker role.

using Microsoft.AzureCAT.Samples.InterRoleCommunication;
using Microsoft.AzureCAT.Samples.InterRoleCommunication.Framework.Configuration;

public class SampleWorkerRole : RoleEntryPoint
{
    private IInterRoleCommunicationExtension interRoleCommunicator;

    /// <summary>
    /// Called by Windows Azure service runtime to initialize the role instance.
    /// </summary>
    /// <returns>Return true if initialization succeeds, otherwise returns false.</returns>
    public override bool OnStart()
    {
        // ...There is usually some code here...

        try
        {
            // Read the configuration settings from app.config.
            var serviceBusSettings = ConfigurationManager.GetSection(ServiceBusConfigurationSettings.SectionName) as ServiceBusConfigurationSettings;

            // Resolve the Service Bus topic that will be used for inter-role communication.
            var topicEndpoint = serviceBusSettings.Endpoints.Get(serviceBusSettings.DefaultEndpoint);

            // Initialize the IRC component with the specified topic endpoint.
            // Instantiating the IRC component inside the OnStart method ensures that IRC component is a singleton.
            // The Singleton pattern prevents the creation of competing consumers for the same IRC events.
            this.interRoleCommunicator = new InterRoleCommunicationExtension(topicEndpoint);
        }
        catch (Exception ex)
        {
            // Report on the error through default logging/tracing infrastructure.
            Trace.TraceError(String.Join(ex.Message, Environment.NewLine, ex.StackTrace));

            // Request that the current role instance is to be stopped and restarted.
            RoleEnvironment.RequestRecycle();
        }

        // ...There is usually some more code here...
        return true;
    }

   // ...There is usually some more code here...
}

In summary, you will need to add a private field of type IInterRoleCommunicationExtension which will be initialized with an instance of the IRC component using the configuration information loaded from app.config.

In web roles, the IRC component initialization logic must be placed into the Global.asax.cs file, most specifically into its Application_Start method. The Application_Start method is called only once during the lifecycle of the IIS application pool that belongs to the web role instance. Another popular approach is to implement a lazy initialization whereby the IRC component instance will only be created when it’s first accessed as shown in the example below:

public class Global : System.Web.HttpApplication
{
    private static Lazy<IInterRoleCommunicationExtension> ircLazyLoader = new Lazy<IInterRoleCommunicationExtension>(() =>
    {
        var serviceBusSettings = ConfigurationManager.GetSection(ServiceBusConfigurationSettings.SectionName) as ServiceBusConfigurationSettings;
        var topicEndpoint = serviceBusSettings.Endpoints.Get(serviceBusSettings.DefaultEndpoint);

        return new InterRoleCommunicationExtension(topicEndpoint);
    },
    LazyThreadSafetyMode.ExecutionAndPublication);

    public static IInterRoleCommunicationExtension InterRoleCommunicator
    {
        get { return ircLazyLoader.Value; }
    }

    // ...There is usually some more code here...
}
ImpNoteNoBgImportant
It is recommended that the IRC component is not to be instantiated by a role instance more than once per single topic. Under the hood, the component maintains a messaging client with an active subscription affinitized to a particular role instance ID. Multiple instances of the IRC component created by a role instance for the same topic will share the same subscription. This will create competing consumers for the subscription and may result in inter-role communication events being consumed by incorrect instances of the IRC component. Therefore, an instance of the InterRoleCommunicationExtension class must always be a singleton for a given topic. Make sure you implement a Singleton pattern when creating an instance of this class. Note that multiple instances of the IRC component in a single role instance are allowed, provided that you initialize the IRC component instances using different topic endpoints.

Step 5: Subscribe to Inter-Role Communication Events

In order to start receiving inter-role communication events, you will need to implement and activate a subscriber to these events. The subscriber is a class implementing the IObserver<InterRoleCommunicationEvent> interface with 3 methods: OnNext, OnCompleted and OnError. The following code is an example of the IRC event subscriber:

/// <summary>
/// Implements a subscriber for inter-role communication (IRC) events.
/// </summary>
public class InterRoleCommunicationEventSubscriber : IObserver<InterRoleCommunicationEvent>
{
    /// <summary>
    /// Receives a notification when a new inter-role communication event occurs.
    /// </summary>
    public void OnNext(InterRoleCommunicationEvent e)
    {
        // Put your custom logic here to handle an IRC event.
    }

    /// <summary>
    /// Gets notified that the IRC component has successfully finished notifying all observers.
    /// </summary>
    public void OnCompleted()
    {
        // Doesn't have to do anything upon completion.
    }

    /// <summary>
    /// Gets notified that the IRC component has experienced an error condition.
    /// </summary>
    public void OnError(Exception error)
    {
        // Report on the error through default logging/tracing infrastructure.
        Trace.TraceError(String.Join(error.Message, Environment.NewLine, error.StackTrace));
    }
}

Once a subscriber is implemented, the next step is to register it with the IRC component by letting the Subscribe method do all the heavy lifting behind the scene:

public class SampleWorkerRole : RoleEntryPoint
{
    private IObserver<InterRoleCommunicationEvent> ircEventSubscriber;
    private IDisposable ircSubscription;

    /// <summary>
    /// Called by Windows Azure after the role instance has been initialized. 
    /// This method serves as the main thread of execution for your role.
    /// </summary>
    public override void Run()
    {
        // ...There is usually some code here...

        // Create an instance of the component which will be receiving the IRC events.
        this.ircEventSubscriber = new InterRoleCommunicationEventSubscriber();

        // Register the subscriber for receiving inter-role communication events.
        this.ircSubscription = this.interRoleCommunicator.Subscribe(this.ircEventSubscriber);

        // ...There is usually some more code here...
    }
}

At this point, everything is ready to go for the very first inter-role communication event to be published. Let’s start with multicasting an IRC event to all active role instances.

Step 6: Publishing Multicast Inter-Role Communication Events

Publishing an IRC event requires creating an instance of the InterRoleCommunicationEvent class with the specified event payload, which can be either a simple .NET type or a custom DataContract-serializable object. The following fragment depicts a simplistic use case in which a string-based payload is packaged as an IRC event and published to all role instances participating in the IRC event exchange:

// Create an instance of the IRC event using a string-based payload.
InterRoleCommunicationEvent e = new InterRoleCommunicationEvent("This is a test multicast event");

// Publish the event. That was easy.
this.interRoleCommunicator.Publish(e);

In most cases, you will be publishing complex, strongly typed events carrying a lot more than a string. That is covered in step 8.

Step 7: Publishing Unicast Inter-Role Communication Events

To publish an IRC event that needs to be received by a specific role instance, you will need to know its role instance ID. If the target role instance resides in another hosted service deployment regardless of the datacenter location, you will also need to be in a possession of its deployment ID. To resolve the ID of a particular role instance that needs to receive an IRC event, you can query the Roles collection and walk through the list of instances for each role to find the recipient’s role instance ID.

The following code snippet is an example of how to send a unicast IRC event to the next instance succeeding the current instance in the role collection:

using System.Linq;

// Find a role instance that sits next to the role instance in which the code is currently running.
var targetInstance = RoleEnvironment.Roles.Where(r => { return r.Value == RoleEnvironment.CurrentRoleInstance.Role; }).
                        SelectMany(i => i.Value.Instances).
                        SkipWhile(instance => { return instance != RoleEnvironment.CurrentRoleInstance; }).
                        Skip(1).
                        FirstOrDefault();

if (targetInstance != null)
{
    // Create an instance of the IRC event using a string-based payload.
    InterRoleCommunicationEvent e = new InterRoleCommunicationEvent("This is a test multicast event", roleInstanceID: targetInstance.Id);

    // Publish the event. That was also easy.
    this.interRoleCommunicator.Publish(e);
}

If an IRC event needs to be received by a role instance that resides outside the current deployment – for instance, in another datacenter or in a different hosted service – you will need to resolve its deployment ID and specify it when creating the InterRoleCommunicationEvent object (see the class constructor’s signature for more details).

ImpNoteNoBgImportant
The Windows Azure Managed Library defines a Role class that represents a role. The Instances property of a Role object returns a collection of RoleInstance objects, each representing an instance of the role. The collection returned by the Instances property always contains the current instance. Other role instances will be available via this collection only if you defined an internal endpoint for the role, as this internal endpoint is required for the role’s instances to be discoverable.

Step 8: Publishing Strongly Typed Inter-Role Communication Events

As noted in step 7, most real-world scenarios for inter-role communication will involve exchanging complex events carrying a lot more than just a single piece of data. For these reasons,  we have enabled you to specify an instance of any custom type as a payload for the IRC events. You will need to decorate your custom type with a DataContract attribute with every member of the type to be marked as serializable using a DataMember attribute unless certain members should not be passed through the wire. Below is an example of a custom type that can be used as an IRC event payload:

/// <summary>
/// Implements an IRC event payload indicating that a new work item was put in a queue.
/// </summary>
[DataContract]
public class CloudQueueWorkDetectedTriggerEvent
{
    /// <summary>
    /// Returns the name of the storage account on which the queue is located.
    /// </summary>
    [DataMember]
    public string StorageAccount { get; set; }

    /// <summary>
    /// Returns a name of the queue where the payload was put.
    /// </summary>
    [DataMember]
    public string QueueName { get; set; }

    /// <summary>
    /// Returns a size of the queue's payload (e.g. the size of a message or the number of messages).
    /// </summary>
    [DataMember]
    public long PayloadSize { get; set; }
}

To publish a strongly typed event, create a new InterRoleCommunicationEvent passing the instance of the custom type and then invoke the Publish method as follows:

var ircEventBody = new CloudQueueWorkDetectedTriggerEvent() { QueueName = "MyQueue" };
var ircEvent = new InterRoleCommunicationEvent(ircEventBody);

this.interRoleCommunicator.Publish(ircEvent);

Strongly typed events require custom types to be shared between publishers and consumers. In other words, the respective .NET types need to be in a possession by the consumer so that it can successfully deserialize the event payload.

ImpNoteNoBgImportant
The maximum size of a serialized IRC event must not exceed the maximum message size allowed by the Service Bus messaging infrastructure. As of writing, this message size limit is 256KB. To stay up-to-date with the latest limits and other important technical constraints that may apply, please visit the Windows Azure Service Bus FAQ on MSDN.

Step 9: Dispose the Inter-Role Communication Component

The implementation of the IRC component relies on messaging APIs provided by the Service Bus. Under the hood, the component instantiates and maintains communication objects which provide access to the underlying messaging infrastructure. These objects require being disposed in a graceful and controlled manner. You will need to ensure that the IRC component is explicitly told to shut itself down by invoking its Dispose method.

The following code sample disposes the IRC component upon termination of the hosting role instance:

public class SampleWorkerRole : RoleEntryPoint
{
    // ...There is usually some code here...

    /// <summary>
    /// Called by Windows Azure when the role instance is to be stopped.
    /// </summary>
    public override void OnStop()
    {
        if (this.interRoleCommunicator != null)
        {
            // Dispose the IRC component so that it gets a chance to gracefully clean up internal resources.
            this.interRoleCommunicator.Dispose();
            this.interRoleCommunicator = null;
        }
    }

    // ...There is usually some more code here...
}

The disposal of the IRC component is guaranteed not to throw any runtime exceptions; hence it’s safe to call into the Dispose method without any special precautions.