Azure Service Bus Queue - how to limit consumption rate to 1 message per minute per consumer instance?

Question

I have 3 consumer instances that read and process messages from the same Azure Service Bus Queue to which producers can send any number of messages at once and overwhelm the very slow consumers.

I want to limit each consumer to read and process only 1 message per minute from the queue. Is there anything in Queue configuration that can limit message rate consumption per minute per consumer instance? Otherwise how can I achieve this?

Answer 1

Azure Service Bus does not natively provide a configuration to limit the message consumption rate on a per-consumer-instance basis. You might be able though to accomplish your objective by using one of the following alternatives:

---

1. Throttling at the Consumer Level Each consumer instance can implement logic to limit the rate of processing. Here’s how:

• Use ReceiveAndDelete or PeekLock Mode:
  • In PeekLock mode, you receive a message and process it within the lock duration.
  • In ReceiveAndDelete mode, you remove the message immediately upon receiving.
• Introduce a Delay:
  • Add a delay between processing messages in your consumer code. For example, after processing a message, wait for 1 minute before fetching the next one.
  • For example (C# with .NET SDK):

        while (true)
        {
            var message = await queueClient.ReceiveAsync(TimeSpan.FromSeconds(5));
            if (message != null)
            {
                // Process the message
                await ProcessMessageAsync(message);
    
                // Complete the message
                await queueClient.CompleteAsync(message.SystemProperties.LockToken);
    
                // Wait 1 minute before processing the next
                await Task.Delay(TimeSpan.FromMinutes(1));
            }
        }
    

---

2. Using a Custom Message Processing Scheduler Deploy an intermediary system (like an Azure Function or a custom service) that pulls messages from the Service Bus Queue and pushes them into a rate-limited queue. The consumers then process messages from the rate-limited queue.

• Steps:
  1. Use an Azure Function or a service that pulls messages from the Service Bus Queue.
  2. Implement rate-limiting logic in this intermediary system.
  3. Use a lightweight message broker (like Redis or an in-memory queue) for the consumers to pull from.

---

3. Alternative with Azure Logic Apps If you prefer a low-code/no-code solution:

• Use Azure Logic Apps to:
  • Pull messages from the Service Bus Queue.
  • Enforce a delay (using a time interval) before forwarding messages to consumers.

---

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin