BizTalk: Rate-limited service calls
Some services have a rate limiter which requires that you call the service no more than, for example, 20 calls during a 10-second period.
This article describes a way to achieve this non-functional requirement. The core code described here is generic and can be used outside BizTalk.
This article also describes how to use the core code from a BizTalk orchestration.
Scenario
You have to call a service, such as an API service, but there are restrictions on the rate by which you can call the service.
In this example, you may only make 20 calls within a 10-second period. That means, if you have made 20 calls and 10 seconds hasn't passed, and you have to make yet another call, you must wait a short period of time before you can make the 21st call.
Solution
The core solution consists of a class with an ordered list with the timestamps of the latest N requests, where N = 20 in the example above. The list is sorted on the timestamp. The class takes care of tracking the requests' timestamps and calculating the waiting time until the next request may be made.
Usage
Public constructor
public RequestRateLimiter(int MaxRequestCount, int duringTimespanSeconds)
The constructor takes the number of requests to keep track of, and the timespan in seconds during which these requests shall be tracked.
Public methods
public void Add()
public TimeSpan GetWaitDuration()
There are two public methods: Add() and GetWaitDuration(). They have overloads if you want to supply your own timestamps, see the Discussion section below on UTC timestamps. The overloads can be used in a unit test to make the tests repeatable.
Typical usage
The typical usage pattern is within a loop where you might not know beforehand how many calls you will make, since that information is retrieved from a previous API call or an external source.
Before you can make a call, you must first see if you need to wait before calling the service, and for how long.
That information is returned from the GetWaitDuration() method: It returns the timespan you have to wait, and, if no wait is necessary, it returns TimeSpan.Zero.
If a time span greater than TimeSpan.Zero is returned, you let your thread sleep for that timespan.
After that, you are allowed to make the service call. Immediately before making the call, you must call the Add() method. That adds the current timestamp to the tracker, and now you're good to go. Make your service call as usual.
Code sample
Here is a sample, using C# code.
RequestRateLimiter rateLimiter = new RequestRateLimiter(20, 10); // maximum 20 requests in 10 seconds
for(int i = 1; i <= 50; i++)
{
TimeSpan waitTime = rateLimiter.GetWaitDuration();
if (waitTime != TimeSpan.Zero)
{
Thread.Sleep(waitTime);
}
rateLimiter.Add();
MyProxy.ServiceAPICall(); //make your call here
}
Complete code
001.using System;
002.using System.Collections;
003.
004.namespace RequestRateLimiterLibrary
005.{
006. /// <summary>
007. /// Class to keep track of a number of requests (actually, only timestamps)
008. /// in order to limit the number of requests made during a sliding time frame.
009. /// A typical requirement is "maximum 20 requests during a 10-second interval".
010. /// Call the Add() method immediately before transmitting a request.
011. /// Before transmitting the next request, call the GetWaitDuration() to determine
012. /// how long your thread must wait until you can transmit the request.
013. /// It is safe to call GetWaitDuration() with an empty history.
014. /// If you supply your own timestamp values, make sure you use UTC timestamps,
015. /// otherwise you may have problems during switch to and from daylight savings
016. /// time.
017. /// Current implementation is not thread-safe. You must call this library from only
018. /// one thread at a time.
019. /// </summary>
020. /// <example>
021. /// RequestRateLimiter hist = new RequestRateLimiter(20, 10);
022. /// for(int i = 1; i <= 50; i++)
023. /// {
024. /// TimeSpan waitTime = hist.GetWaitDuration();
025. /// if (waitTime != TimeSpan.Zero)
026. /// {
027. /// Thread.Sleep(waitTime);
028. /// }
029. /// hist.Add();
030. /// //make your call here
031. /// }
032. /// </example>
033. [Serializable]
034. public class RequestRateLimiter
035. {
036. private SortedList history;
037.
038. private int maxRequestCount;
039. private TimeSpan timeframe;
040.
041. /// <summary>
042. /// Initializes and configures the rate limiter.
043. /// Typical scenario is "maximum 20 requests during 10 seconds".
044. /// </summary>
045. /// <exception cref="ArgumentException">Throws ArgumentException if any constructor parameter is smaller than 1.</exception>
046. /// <param name="MaxRequestCount">The number of request timestamps to keep track of.</param>
047. /// <param name="duringTimespanSeconds">The time span for which the MaxRequestCount must be maintained.</param>
048. public RequestRateLimiter(int MaxRequestCount, int duringTimespanSeconds)
049. {
050. if (MaxRequestCount < 1 || duringTimespanSeconds < 1)
051. {
052. throw new ArgumentException("Both MaxRequestCount and duringTimespanSeconds must be greater than zero.");
053. }
054. this.history = new SortedList();
055. this.maxRequestCount = MaxRequestCount;
056. this.timeframe = new TimeSpan(0, 0, duringTimespanSeconds); // hours, minutes, seconds
057. }
058.
059. /// <summary>
060. /// Add a timestamp (of when the request is made).
061. /// If the maximum number of requests is already reached,
062. /// the oldest timestamp is removed.
063. /// </summary>
064. /// <param name="currentUTC">The timestamp for the request. If ignored or null, the computer's current UTC date and time will be used.</param>
065. public void Add(DateTimeOffset? currentUTC = null)
066. {
067. if (this.history.Count >= this.maxRequestCount)
068. {
069. // index is zero-based
070. this.history.RemoveAt(0);
071. }
072. if (currentUTC == null) { currentUTC = DateTimeOffset.UtcNow; }
073. this.history.Add(currentUTC, currentUTC);
074. }
075.
076. /// <summary>
077. /// Add a timestamp of the computer's current UTC date and time.
078. /// If the maximum number of requests is already reached,
079. /// the oldest timestamp is removed.
080. /// </summary>
081. public void Add()
082. {
083. Add(null);
084. }
085.
086.
087. /// <summary>
088. /// Calculates the waiting time to fulfill the requirement of
089. /// a maximum number of requests during a sliding time span.
090. /// If no waiting is necessary, Timespan.Zero is returned.
091. /// It is safe to call GetWaitDuration() with an empty history.
092. /// </summary>
093. /// <param name="currentUTC">The UTC date and time for which to calculate the wait time. If ignored or null, the computer's current UTC date and time will be used.</param>
094. /// <returns>A TimeSpan, possibly TimeSpan.Zero if no wait is needed</returns>
095. public TimeSpan GetWaitDuration(DateTimeOffset? currentUTC = null)
096. {
097. if (this.history.Count < this.maxRequestCount)
098. {
099. return TimeSpan.Zero;
100. }
101. // index is zero-based
102. DateTimeOffset newest = (DateTimeOffset)this.history.GetByIndex(this.history.Count - 1);
103. DateTimeOffset oldest = (DateTimeOffset)this.history.GetByIndex(0);
104.
105. if (currentUTC == null) { currentUTC = DateTimeOffset.UtcNow; }
106.
107. // Back one time frame and see where we end up:
108. // Type casting needed to cast from DateTimeOffset? to DateTimeOffset
109. DateTimeOffset oneTimeframeBack = (DateTimeOffset)currentUTC - this.timeframe;
110. // If we end up before the oldest, we have to wait until the oldest will be outside the time frame:
111. //
112. // ____I_I_I_I_I_I_I_I_I_I_I_I_I_I_I_I_I_I_I_I________________> t
113. // ^ ^ ^now ^
114. // | | |
115. // |oldest |newest |(oldest+timeframe)
116. //
117. if (oneTimeframeBack < oldest)
118. {
119. return oldest + this.timeframe - (DateTimeOffset)currentUTC;
120. }
121. else
122. {
123. // The oldest was older than a time frame, that means we can transmit immediately:
124. return TimeSpan.Zero;
125. }
126. }
127.
128.
129. /// <summary>
130. /// Calculates the waiting time to fulfill the requirement of
131. /// a maximum number of requests during a sliding time span.
132. /// If no waiting is necessary, Timespan.Zero is returned.
133. /// It is safe to call GetWaitDuration() with an empty history.
134. /// </summary>
135. /// <returns>A TimeSpan, possibly TimeSpan.Zero if no wait is needed</returns>
136. public TimeSpan GetWaitDuration()
137. {
138. return GetWaitDuration(null);
139. }
140. }
141.}
The complete code can be downloaded from gitHub.
Discussion
UTC Timestamps
The timestamps must be made in UTC time, otherwise you will have problems when local time changes during entering and leaving daylight savings time. The code is not safe against changing the time, since that will change the UTC time too.
Serializable
Note that the class is marked [Serializable]. It will make an instance of the class serializable to storage, a function needed when using the class in a BizTalk orchestration.
A BizTalk orchestration (see below) needs to serialize itself, and any used objects, to database when it reaches a persistence point. Examples of persistence points are when it sends a message, waits for a response, and when suspending an orchestration instance because of an error.
Singleton pattern
This class was designed for one rate limiter per session (login call returns a session ID, subsequent calls use the session ID, and finally a logout call). In the case where the rate limit is per IP address, you might consider using a Singleton pattern, and maybe on one server only if you have a BizTalk cluster.
Application of solution within BizTalk
To use this solution from within a BizTalk application, you must use an orchestration (another, but more far-fetched possibility is to code this into a new adapter).
The sample here is basically the same as the C# sample above – it receives a request, performs a service call 50 times and then responds – only coded with BizTalk shapes:
After receiving the request, it initializes the rate limiter with the desired rate limit values.
Then it initializes the loop variables, and enters the loop. Inside the loop, it constructs the service request message.
Before making the service call, it checks if the limit is reached by calling the rate limiter's GetWaitDuration() method.
If the returned time span is something else than TimeSpan.Zero, it delays the current orchestration thread with that time span.
If the returned time span is TimeSpan.Zero, it means the rate limit is not reached yet and no delay is necessary.
Immediately before the orchestration sends the message, the orchestration must register that a call is made with the Add() method.
Then, the message is sent and the orchestration awaits a reply as usual. When the loop is finished, a response message is created and returned to the caller.
In a real-world scenario, you will have to handle errors, but such code is left out here, to keep the sample easy to follow. The same applies to any request message creation, mapping, and handling reply messages, which pertains to your actual business scenario.
When using BizTalk, there will be additional small delays when BizTalk stores the message to the message box, and for the send adapter to pick it up. However, those delays are usually small compared to limits, and will help you stay below the limits.
See also
Singleton orchestrations:
Implementing Singleton pattern with BizTalk Orchestrations.
Another important place to find a huge amount of BizTalk related articles is the TechNet Wiki itself.
The best entry point is BizTalk Server Resources on the TechNet Wiki.