HPC Java Bridge - Enabling Java-to-Java Communication

HPC Server 2008 R2 SDK SP1 contains sample code that demonstrates the usage of Java on the client side. The framework enables a Java-to-Java communication within HPC Server 2008 R2 SP1. It enables to send Java objects on the client side and process them on the service side within stateful Java processes. The Project Sources are published on MSDN.

Architectural Overview - How Does it Work?

The HPC Java Bridge plugs into the existíng HPC SOA architecture. We can identify at least 3 different tiers that are related to the bridge:

• The Java Client: On the client side a programmer can serialize Java objects that are sent via the bridge to the Java service side. The client uses the Java API parts of the proposed framework to create a session pool based on durable or non-durable sessions. The framework further allows to transfer the serialized messages in a pretty easy way to the Java service side. The client API parts contain failover handling and a support of large message transfers. Processed messages or exceptions are forwarded asynchronously.
• The HPCJavaBridge.dll: The HPCJavaBridge is a standard WCF service that acts as a gateway for Java objects. The service spawns Java processes that implement the service parts of the Java API and controls their lifecycles. It keeps the spawned Java processes up and running for the lifetime of the WCF service instance.
• The Java Service: On the service side a programmer needs to extend the service parts of the proposed Java API. Lifetime and exception handling of the code are handled by the framework.

Configure the WCF Service

Please follow these steps to configure the WCF Service.

1. Create a new WCF service configuration file called HPCJavaBridge.config or edit the one provided in the Project Sources. Edit the ServiceRegistration section of the configuration file and insert the path to the HPCJavaBridge.dll assembly.
2. Specify the ChunkFolder location: Serialized Java objects > 3 MB (durable mode) or > 35 MB will be chopped into chunks. The chunks must be stored on a UNC path location.
3. Specify the location of java.exe on the compute nodes using the JavaExe environment variable. The WCF Service will spawn a Java process and forward the incoming serizalized Java objects.
4. Specify the Java arguments using the JavaArgs environment variable. This is the location where to configure the Java classpath and the Java Service entry point. In the provided sample it's the class hpc.test.PingPongSampleService. Do not add the Apache CXF libraries to the specified classpath.
5. Specify the PortRangeStart and the PortRangeEnd environment variables. The spawned Java process will listen to a random port within the port range on localhost of the executing compute node. Make sure that the firewall settings on the compute nodes won't disturb this communication gateway.
6. Run the SOA Service Loading Test against the new WCF service from the Cluster Manager console.

<microsoft.Hpc.Session.ServiceRegistration>    <service assembly="\\myshare\HPCJavaBridge\HPCJavaBridge\bin\Debug\HPCJavaBridge.dll"      contract="HPCJavaBridge.IHPCJavaBridge"      type="HPCJavaBridge.HPCJavaBridge" includeExceptionDetailInFaults="true"      maxConcurrentCalls="0" serviceInitializationTimeout="60000"      stdError="" maxMessageSize="2147483647">      <environmentVariables>          <add name="ChunkFolder" value="\\myshare\Chunks\"/>  <add name="JavaExe" value="\\myshare\jdk1.6.0_21\jre\bin\java"/>  <add name="JavaArgs" value="-Xms64m -Xmx1024m -cp \\myshare\workspace\HPCJavaService\bin\. hpc.test.PingPongSampleService"/>  <add name="PortRangeStart" value="42000"/>  <add name="PortRangeEnd" value="49000"/>      </environmentVariables>    </service></microsoft.Hpc.Session.ServiceRegistration>

Implementing a Simple Client

The Java client side requires the CXF libraries as well as the generated Session and HPC Java Bridge service API. Once added to the classpath create custom Java client code as displayed below. It allows compression, serialization and distribution of any serializable Java object through the JavaClientSingleton class. 

Let's have a look at the following SimplePingPongClient sample: 

• Creating a session in order to connect to a SOA service in HPC Server 2008 R2 requires at least the specification of the name of the head node, the SOA service name and the AD user details. The JavaClientSingleton class further allows to specify a job template name (if not it will use the default job template) and the minimum and maximum required cores. The session share feature allows multiple broker clients to submit their messages through the same session. This speeds up the communication between the client and the service because the session creation time is quite long compared to the submission time of a message. If a durable behavior is required, the JavaClientSingleton will create durable sessions instead of non-durable ones.
• If messages or exceptions from the service side are received, the modified session API of HPC Server 2008 R2 will invoke the OnMessageReceive-method of an AsyncMessageReceiver object. The object received is already de-compressed and de-serialized.
• To send messages from the Java client to the Java service side the JavaClientSingleton class provides the distributeJobs-method that compresses and sends an array of serializable object to a(n existing) session.

package hpc.test;

import com.microsoft.hpc.scheduler.session.modified.AsyncMessageReceiver;
import hpc.clientSide.JavaClientSingleton;
import hpc.sharedObjects.PingPong;

public class SimplePingPongClient {

 public static void main(String[] args) throws Exception {
   JavaClientSingleton jc = JavaClientSingleton.getInstance();
jc.setHeadNode("head");
jc.setDomainName("hpc");
jc.setUserName("alibaba");
jc.setPassword("al!b!ba");
jc.setServiceName("HPCJavaBridge");
jc.setJobTemplateName("hpcjavabridge");
jc.setMinUnits(2);
jc.setUnitType(0);
jc.setMaxUnits(4);
jc.setReuseSession(true);
jc.setUseDurableSessions(false);

AsyncMessageReceiver amr = new AsyncMessageReceiver() {
      @Override public void OnMessageReceive(Object Message) {
System.out.println("[received]");
          if(Message instanceof PingPong)
{
PingPong pp = (PingPong) Message;
             System.out.println("ping/pong from host: " + pp.getHostName());
}
}
};

jc.distributeJobs(new Object[] {new PingPong(), "I'm a string job", new Long(123)}, 4, amr, true);
}
}

 

Method	Description
setHeadNode	sets the name of the cluster head node
setDomainName	sets the name of the users domain
setUserName	sets the username
setPassword	sets the users password
setServiceName	sets the service name under which the HPCJavaBridge got configured
setJobTemplateName	sets the name of the job template. If non is specified it will use the Detault job template
setMinUnits	sets the guaranteed minum amount of units. The value must be > 0
setMaxUnits	sets the maximum amount of units. The value must be bigger or equal than the minimum amout of units
setUnitType	sets the unit type to allocate on the cluster. Allowed values are: 0 = core, 1 = socket, 2 = node
setReuseSession	if set to true sessions won't be closed if possible. This reduces the the send/response time for new job distributions
setUseDurableSessions	if set to true only durable sessions will be used for job distribution. In case of a failover the distributed jobs won't be lost
distributeJobs	distributes the job objects. Parameters are: Object[] jobs: the array of serializable job objects that will be sent to the cluster int priority: the session priority. Allowed values range from 0 (= lowest priority) to 4 (= highest priority) AsyncMessageReceiver clientCallback: the async callback object boolean waitForCompletion: if set to true the method won't return before all distributed jobs responses are collected
warmUp	Warms up a session by sending simple ping pong messages to the cluster. This will create a session and initialize the service instances which consumes most of the time. By calling this method at the beginning of all user interactions the minimum send/response time is achieved. Parameters are: int priority: the session priority. Allowed values range from 0 (= lowest priority) to 4 (= highest priority) AsyncMessageReceiver clientCallback: the async callback object boolean waitForCompletion: if set to true the method won't return before all distributed jobs responses are collected
closeAllSessions	Closes all sessions.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Implementing the Service Side

 The published service API includes a ping pong test class. Any Java service that will be implemented to interchange with the proposed HPC Java Bridge framework will need to extend the JavaProcessStarterImpl class. Place your custom code with these 3 methods:

• The initializeInstance-method is called first by the HPC Java Bridge framework when the Java process got spawned by the WCF service. Place all initialization code in here since it will be called only once per JVM instance.
• The shutdownInstance-method is called first by the HPC Java Bridge framework when the associated SOA session is finishing. Place all instance cleaning code in here since it will be called only once per JVM instance.
• The processInstance-method is called every time a Java object is arriving from the client side. Place your custom message processing code in here.

Make sure to add a main-method to your starting class. Call the Publish-method to startup the Java process to start the communication with the HPC Java Bridge framework. The WebService attribute values need to be identical for every custom service. Make sure to copy and past them from the PingPongSampleService class.

package hpc.test;

import hpc.serviceSide.fileLessIntegration.JavaProcessStarterImpl; import java.net.InetAddress; import java.util.Date;
import javax.jws.WebService; @WebService( endpointInterface = "hpc.serviceSide.fileLessIntegration.JavaProcessStarter", name="JavaProcessStarter", serviceName="JPSService", targetNamespace=https://hpcjavasoabridge.org/soa2java)
public class PingPongSampleService extends JavaProcessStarterImpl {
@Override protected void initializeInstance() { System.out.println("local init");
}
@Override protected void shutdownInstance() throws Exception { System.out.println("good-bye!");
} @Override protected Object processInstance(Object JavaObject) throws Exception { System.out.println("processing object");
if(JavaObject instanceof Long)
return "hello long";
else
return new Date();
}

  public static void main(String[] args) throws Exception {
PingPongSampleService PPSS = new PingPongSampleService();
PPSS.Publish(args);
}

}

 

Performance

The performance tests are ongoing. First results with sending, compressing, receiving and decompressing simple PingPong Java objects within a "warm" session show very, very fast results.

Comments

• Anonymous
  January 01, 2003
  The files were deleted. The article refers to SP1. Please consider using SP3 regarding Java support. Java Libs are available @ github.com/MicrosoftHPC

• Anonymous
  January 03, 2012
  The hpc.clientSide sources are not available in the downloads. The link name says Hpclientapisources but it downloads hpc.dotnetjavabridge. Where can I find the hpc.clientSide classes?