Month: March 2020

Posted on

Ensuring Ordered Delivery of Messages using Azure Functions

Typically when using Azure Functions to consume messages from a Service Bus (SB), the ordering is not guaranteed although the SB is First-In-First-Out (FIFO). This is due to competing consumers, where multiple instances of the function are competing for messages of the service bus.

An example where out of ordering can happen is when a function instance takes longer to process a message than other instances therefore affecting the process ordering. This is represented in the sequence diagram below, where the function instance 1 took longer to update the same record in a database than instance 2.

image

One option to enforce ordered delivery is to configure the Azure Function to spin up only one instance. The only problem with this solution is it won’t scale very well. A more scalable option is to use sessions.This allows you to have multiple instances of a function executing giving you a higher message throughput.

To enforce message ordering several properties must be set. The property Requires Session must be enabled on the SB queues and topic subscriptions. Messages sent onto the SB must set the context property SessionId to unique value from other non related messages. Some examples of a session Id could be the account number, customer number, batch Id, etc. Azure Functions need to have the IsSessionsEnabled property set to enabled on the SB input binding.

This feature for Azure Functions to use SB sessions only came GA as of mid 2019. Enabling sessions on the Azure Function places a lock on all messages that have the same session Id causing the locked messages to be consumed by that one function instance that placed the lock.

Typical Scenario

A warehouse needs to track the progress of an order from when its first received to when it gets dispatched. Throughout each stage (Ordered, Picked, Packaged, Dispatched) of the ordering process, the status of the order must be updated. This involves placing a new message onto the service bus every time the order status needs to get updated. An Azure function will then pull the messages from the service bus and update the order status in a database where the customer can view the current state of their order.

To simulate the warehouse tracking system, a console app will be used to create messages for each status change (Ordered, Picked, Packaged, Dispatched), for several hundred orders. The session Id of each status message will be set to the order number. The app will then send the messages to a SB Topic where it will have two subscriptions, one with sessions enabled and the other disabled. This is so we can compare the ordering of messages being received with and without sessions enabled.

Order message generater
  1. class Program
  2.   {
  4.       private static string connectionString = ConfigurationManager.AppSettings[“ServiceBusConnectionString”];
  5.       private static string topicName = ConfigurationManager.AppSettings[“TopicName”];
  6.       private static int orders = 100;
  7.       private static int messagePerSession = 4;
  9.       static async Task Main(string[] args)
  10.       {
  11.           Console.WriteLine(“Creating Service Bus sender….”);
  12.           var taskList = new List<Task>();
  13.           var sender = new MessageSender(connectionString, topicName);
  15.           //create an order
  16.           for (int order = 0; order < orders; order++)
  17.           {
  18.               var orderNumber = $”OrderId-{order.ToString()}“;
  19.               var messageList = new List<Message>();
  20.               //simulate a status update in the correct order
  21.               for (int m = 0; m < messagePerSession; m++)
  22.               {
  23.                   var status = string.Empty;
  24.                   switch (m)
  25.                   {
  26.                       case 0 :
  27.                           status = “1 – Ordered”;
  28.                           break;
  29.                       case 1:
  30.                           status = “2 – Picked”;
  31.                           break;
  32.                       case 2:
  33.                           status = “3 – Packaged”;
  34.                           break;
  35.                       case 3:
  36.                           status = “4 – Dispatched”;
  37.                           break;
  38.                   }
  40.                   var message = new Message(Encoding.UTF8.GetBytes($”Status-{status}“))
  41.                   {
  42.                       //set the service bus SessionId property to the current order number
  43.                       SessionId = orderNumber
  44.                   };
  45.                   messageList.Add(message);
  46.               }
  48.               //send the list of status update messages for the order to the service bus
  49.               taskList.Add(sender.SendAsync(messageList));
  50.           }
  52.           Console.WriteLine(“Sending all messages…”);
  53.           await Task.WhenAll(taskList);
  54.           Console.WriteLine(“All messages sent.”);
  55.       }
  56.   }

Two Azure functions will be created, where one has sessions enabled and the other disabled. The functions will have a random delay created from 1 to 10 seconds to simulate some business logic which may be calling out to an external service before updating the order status. Instead of the function writing to a database, each status update message received will be written to an Azure Table storage to create an audit log of when a status update message was processed.

Below is the source code for the function which will process the messages on the service bus using sessions. Note the IsSessionEnabled property is set to true on the ServiceBusTrigger input binding. The randomiser is to simulate some business logic that could vary in time to process a message.

Azure Function using sessions
  1. public static class MsgOrderingSessions
  2.     {
  3.         [FunctionName(“MsgOrderingSessions”)]
  4.         [return: Table(“OrdersSession”, Connection = “StorageConnectionAppSetting”)]
  5.         public static OrderEntity Run([ServiceBusTrigger(“orders”, “OrdersSession”, Connection = “SbConnStr”, IsSessionsEnabled = true)]
  6.               Message sbMesssage, ILogger log)
  7.         {
  8.             log.LogInformation($”C# ServiceBus topic trigger function processed message: {Encoding.UTF8.GetString(sbMesssage.Body)}“);
  10.             Random random = new Random();
  11.             int randNumb = random.Next(1000, 10000);
  12.             System.Threading.Thread.Sleep(randNumb);
  14.             return new OrderEntity { PartitionKey = $”{sbMesssage.SessionId} – {DateTime.Now.Ticks}“ , RowKey = Guid.NewGuid().ToString(), Text = Encoding.UTF8.GetString(sbMesssage.Body) };
  16.         }
  18.     }

Below is the source code for the function which does not use sessions. Here the IsSessionEnabled is set to false.

Azure Function no sessions
  1. public static class MsgOrderingNoSession
  2.     {
  3.         [FunctionName(“MsgOrderingNoSessions”)]
  4.         [return: Table(“OrdersNoSession”, Connection = “StorageConnectionAppSetting”)]
  5.         public static OrderEntity Run([ServiceBusTrigger(“orders”, “OrdersNoSession”, Connection = “SbConnStr”, IsSessionsEnabled = false)]
  6.               Message sbMesssage, ILogger log)
  7.         {
  8.             log.LogInformation($”C# ServiceBus topic trigger function processed message: {Encoding.UTF8.GetString(sbMesssage.Body)}“);
  10.             Random random = new Random();
  11.             int randNumb = random.Next(1000, 10000);
  12.             System.Threading.Thread.Sleep(randNumb);
  14.             return new OrderEntity { PartitionKey = $”{sbMesssage.SessionId} – {DateTime.Now.Ticks}“, RowKey = Guid.NewGuid().ToString(), Text = Encoding.UTF8.GetString(sbMesssage.Body) };
  16.         }       
  17.     }

Below is the settings for the service bus topic which has 2 subscriptions and one of them has Requires Session checked.

image

Running the console app creates 400 messages on both subscriptions, 4 status update messages per 1 order.

image

Conclusion

The Azure function which had the ServiceBusTrigger, IsSessionsEnabled = false inserted the rows out of order due to multiple function instances competing for the next message on the service bus.

image

Now the Azure Function which had IsSessionsEnabled = true and read messages from a service bus subscription which also had the Requires Session flag enabled, the messages were processed in the correct sequence as they were placed onto the service bus.

image

When using sessions, there is a slight performance hit depending on the number of function instances executing. In this example both functions where running under the consumption plan which spun up 6 instances. As you can see the number of messages waiting on each of the subscriptions below, the subscription which had sessions disabled are processing the messages a lot faster.

When sessions are used, each function instance places a locked on all messages having the same session Id which are processed one after another. As there were only 6 instances available, only a maximum of six orders could be processed at one time.

image

Enjoy…