HPC Pack SOA Tutorial - Batch mode

In the previous tutorial, you created a simple SOA client and service. In most cases, the HPC algorithm is not as simple as adding two numbers. Some services are more complex and run for hours.

For such services, the user normally submits all the requests, then retrieves the responses in a couple of hours. This represents a challenge for developers. Can you close the client application after sending all the requests, and then start another client to get the responses the next day? This article shows you how to design the application to handle such scenarios.

The key requirements of the batch mode is to make sure the requests won't be lost after being sent and that the results won't be lost.

This article creates a new service to explain how to create such reliable computations. This service does prime factorization. A client application submits 100 big numbers to the service. The service starts another client application to retrieve the results.

Implement the service

Download the accompanying code sample to build the service.

Here is the service contract:

[ServiceContract]
public interface IPrimeFactorization
{
    [OperationContract]
    List<int> Factorize(int n);
}

Here is the service implementation:

public List<int> Factorize(int n)
{
    List<int> factors = new List<int>();
    for (int i = 2; n > 1; )
    {
        if (n % i == 0)
        {
            factors.Add(i);
            n /= i;
        }
        else
        {
            i++;
        }
    }
    return factors;
}

Implement the client to send requests

HPC Pack provides a feature called DurableSession. It makes sure all the requests and the responses are persisted.

Write a client application to send the requests.

1. First we need to prepare the session information, as in the previous tutorial.

   SessionStartInfo info = new SessionStartInfo("head.contoso.com", "PrimeFactorizationService"); 
   

2. Create a DurableSession object, like the following code:

   //Create a durable session
   DurableSession session = DurableSession.CreateSession(info);
   Console.WriteLine("Session {0} has been created", session.Id);
   

3. Create a BrokerClient object to send requests.

   //Send batch request
   Random random = new Random();
   const int numRequests = 100;
   
   using (BrokerClient<IPrimeFactorization> client = new BrokerClient<IPrimeFactorization>(session))
   {
       Console.WriteLine("Sending {0} requests...", numRequests);
       for (int i = 0; i < numRequests; i++)
       {
           int number = random.Next(1, Int32.MaxValue);
   
           FactorizeRequest request = new FactorizeRequest(number);
   
           //The second param is used to identify each request.
           //It can be retrieved from the response. 
           client.SendRequest<FactorizeRequest>(request, number);
       }
   
       client.EndRequests();
       Console.WriteLine("All the {0} requests have been sent", numRequests);
   }
   

   Call EndRequests() to indicate that all requests have been submitted. After calling EndRequests(), you cannot use this client object to send more requests.

   The application is complete. Because we used a durable session, it's fine to close the client. All the requests are still on service side and are calculated by the compute nodes in the cluster.

Monitor the progress

In HPC Cluster Manager, you can monitor the progress of the requests submitted. Go to Job Management to find your job, and select View Job. You can see how many requests have been processed.

Implement the client to retrieve responses

After a couple of hours, the user can come back and retrieve the results.

1. Prepare the session information. You need the session ID of the previous session to help the client to attach the session.

   //Input sessionId here
   int sessionId;
   Console.Write("Input the session id : ");
   sessionId = Int32.Parse(Console.ReadLine());
   
   //Change the headnode name here
   SessionAttachInfo info = new SessionAttachInfo("head.contoso.com", sessionId);
   

2. Instead of creating a new session, attach to the existing session.

   //Attach to session
   DurableSession session = DurableSession.AttachSession(info);
   

3. Get responses from the session.

   //Get responses
   using (BrokerClient<IPrimeFactorization> client = new BrokerClient<IPrimeFactorization>(session))
   {
       foreach (BrokerResponse<FactorizeResponse> response in client.GetResponses<FactorizeResponse>())
       {
           int number = response.GetUserData<int>();
           int[] factors = response.Result.FactorizeResult;
           Console.WriteLine("{0} = {1}", number, string.Join<int>(" * ", factors));
       }
   }
   

4. Close the session.

Keep in mind

• Because of persistence of requests and responses, the performance of the durable session is slightly slower than that of the interactive session.

• session.Close(true) purges all the persisted data related to this session. It should only be called when the session and its data are not needed any longer. If the session will be used going forward, call session.close(false).

• If EndRequests() is not called, the service keeps waiting for the upcoming request. After a period of time, by default, 5 minutes, it removes the client if it does not receive more requests. All the requests sent by this client are be removed.

Next steps

HPC Pack SOA Tutorial – Interactive mode