Simulating 1000 agents using ACS, ACI, and Durable Functions - (1) ACI
I did some experiment to simulate 1000 agents using several measures. I'd like to share some learning and code in this post.
Problem Overview
I wanted to simulate 1000 agents with cost effective way with the simplest solution. Also, wanted to use .Net based technology as much as possible. In this case, How can we use Azure technologies?
On this experiment, I use ACI, ACS (Kubernetes with Go lang), Durable Functions. This system is simple. Just want to printout when we've got a queue. They have 1000 client in on Premise. If possible we want to achieve less than 1 sec latency from sending message to the queue to store print state to the Storage Table.
The architecture is also simple. When customer want to print on a web server, send a message to Storage Queue. Then Azure Functions create data on a table storage. Print Agents are always polling the table storage. Once get a data, it print the data and sends a queue to Storage Queue. Then Azure Functions store the print status to the queue.
Simulator Candidates
Which technologies to use for this use case? Using a container is one idea. We use .Net framework this time, We convert it to the .NetCore then pack it using Visual Studio. You can refer Visual Studio Tools for Docker with ASP.NET Core. We can use Azure Container Instances or Azure Container Service. Also, Durable Functions might be good since it has good orchestrator. I'm not sure if it fits for this use case, however, it worth trying. I can learn something at least.
Packing a docker image
Once you use Visual Studio Tools for Docker, Visual Studio automatically generate proper Dockerfile like this.
Dockerfile
FROM microsoft/dotnet:2.0-runtime AS base
WORKDIR /app
FROM microsoft/dotnet:2.0-sdk AS build
WORKDIR /src
COPY *.sln ./
COPY ClientAppNetCore/ClientAppNetCore.csproj ClientAppNetCore/
COPY CommonNetCore/CommonNetCore.csproj CommonNetCore/
RUN dotnet restore
COPY . .
WORKDIR /src/ClientAppNetCore
RUN dotnet build -c Release -o /app -c ClientAppNetCore.csproj
FROM build AS publish
RUN dotnet publish -c Release -o /app -c ClientAppNetCore.csproj
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "ClientAppNetCore.dll"]
It is using multi staging build, well created. All we need is to name it and push to public/private repository. If you want to push this one, move to the root directory which have *.sln file.
$ docker build . -f ClientAppNetCore/Dockerfile -t tsuyoshiushio/client:1.0.0 -t tsuyoshiushio/client:latest
$ docker push tsuyoshiushio/client:1.0.0
$ docker push tsuyoshiushio/client:latest
Now you are ready to use this agent simulator using docker. 'tsuyoshiushio' is my DockerHub repository name.
Azure Container Instance
This is the Serverless docker hosting service. If it works, it might be a very good solution. Since this client requires different environment variables for one by one containers, I need some code to control the containers. You can refer the whole source code on my GitHub. You can create containers quite easily.
First, Getting credentials for client
var credentials = SdkContext.AzureCredentialsFactory.FromServicePrincipal(clientId, clientSecret, tenantId, AzureEnvironment.AzureGlobalCloud);
var client = new ContainerInstanceManagementClient(credentials);
client.SubscriptionId = subsctiptionId;
Then configure container. Especially specify the resource you need.
var resources = new ResourceRequirements();
var request = new ResourceRequests();
request.Cpu = 0.2;
request.MemoryInGB = 0.3; // once I tried 0.1 it cause a long execution time.
resources.Requests = request;
var list = new List<Container>();
for (int i = 0; i < 5; i++)
{
list.Add(CreateClientContainer("DOCKERHUB_NAME/client", i, resources, $"{guid}-{i}"));
}
Then create an instance. On the actual code, I needed to deploy one more client called spammer. It sends a lot of queue. Once finished, I wanted to terminate. That is why I specify RestorePolicy = "Never".
await client.ContainerGroups.CreateOrUpdateAsync(resourceGroup, containerGroup,
new ContainerGroup()
{
Containers = list,
Location = location,
OsType = "Linux",
RestartPolicy = "Never" // try not to emit the message by restarting
}
);
This must be the best solution for this time! Very easy! However, it wasn't I can't generate enough performance for this. Since this is preview, we have some limitation.
By default, Container Group limitation is 20 for a subscription. Number of container per container group is 60. We can deploy 1200 containers by default. The biggest problem is container create 60 per hour. I can't create 1000 container. obviously. I needed to contact support service to expand the quotas. However, I'm not sure if we can try 1000 containers since this is preview. This is totally great for "Serverless" solution. However, I needed to give up this time.
However, operating ACI with C# is very easy! We can use it for a lot of purposes!
Conclusion
ACI interface is very good and easy to operate using Azure SDK. However, it have some quota limitation. That is why, I give up this time. However ACI is very good tool to create a some containers with serverless architecture!
I ask the support to expand it, if I can do it, I'll edit this blog.
On the next blog, I'm talking about the k8s with go lang strategy to achieve this issue.