FAQs, Best Practices, Issues and Workarounds
Frequently Asked Questions
| Can I Use “Cloud Numerics” with F#? |
| Can Multiple Cloud Numerics Applications Use the Same Azure Storage Account? |
| Can I Share an Existing “Cloud Numerics” Solution with Another User? |
Best Practices
| Memory Usage and Matrix Size Guidelines |
| Partition the Use of Memory Resources in Your Program with using{} |
Known Issues
Frequently Asked Questions (FAQs)
Can I Use “Cloud Numerics” Libraries and Runtime with F#?
Yes. See the blog post titled “Cloud Numerics” Example: Distributed Numerics on Azure with F#.
Can Multiple Applications Use the Same Windows Azure Storage Account?
Yes, but you need to ensure that these applications have distinct project names, otherwise, each will overwrite the other application’s output. The post build event of a Microsoft codename “Cloud Numerics” C# project uses the project name to create an Azure blob. Identical project names using the same Windows Azure storage account will map application output to the same blob.
If multiple users use the same Windows Azure storage account while creating “Cloud Numerics” applications for Azure by way of the “Cloud Numerics” Deployment Utility, the applications must use different project names.
To share your “Cloud Numerics” Visual Studio solution with another user, a small modification must be made to the solution’s properties in order for the other user to execute the compiled solution locally. The reason for this is that the solution has fully-qualified paths in its application properties, and these paths reflect the directory structure of the development machine where the application was originally developed. For example, the following screen shows fully-qualified directory paths that descend into a user’s home directory.
To change these path specifications so that a different user can run the compiled solution locally on a different machine, follow these steps.
| Note! |
Applications submitted to be run on Windows Azure can be shared without this modification. |
To change the Start Option paths to reflect your local machine:
- From Solution Explorer, right-click the MSCloudNumericsApp and select Properties.
- Click the Debug tab.
- In the Start Options section of the Debug tab, edit the following fields to reflect the paths on your local development machine:
a. For the Command line arguments field, change the path to reflect the directory on the current machine where the “Cloud Numerics” Projects directory can be found.
For example, given the figure above, you would change:
c:\users\roastala\documents\visual studio 2010\Projects…to:
c:\users\<YourUsername>\documents\visual studio 2010\Projects…b. For the Working directory field, change the path to reflect the directory on the current machine where the “Cloud Numerics” Projects directory can be found.
For example, given the figure above, you would change:
c:\users\roastala\documents\…
to:
c:\users\<YourUsername>\documents\…--Where c:\users\<YourUsername>\ reflects the home directory of the user on the local development machine where you put the “Cloud Numerics” project.
4. Change the Project Configuration from Debug to Release, and repeat the changes for the Release configuration.
Best Practices
Memory Usage and Matrix Size Guidelines
The recommended maximum memory usage is <=30% of physical memory. For a Windows Azure node used by a “Cloud Numerics” job, this equals roughly 4GB per node. All active variables in your application should not consume more than 30% of the physical memory available on the cluster. By default, the “Cloud Numerics” Deployment Utility creates clusters with extra-large Windows Azure nodes (8 cores, 14 GB memory).
Partition the Use of Memory Resources in Your Program with using{}
For improved performance, you can evaluate any memory intensive array operation such as matrix creation and cloning, matrix multiplication, LU decomposition, or SVD decomposition within the C# using{} construct. This construct forces the de-allocation of memory from the managed heap when the program leaves the scope of execution bounded by the using{}. If you do not exercise using{} or similar constructs to free unused allocated memory, your cluster can run with noticeable latency and long processing times.
Possible Issues and Suggested Workarounds
Transport-Level Error When Receiving Results from the Server
If you see the following error when interacting with SQL Azure, shut down all connections with SQL Azure, wait 10 to 20 minutes, and attempt your transaction again.
“A transport-level error has occurred when receiving results from the server (provider: TCP Provider, error:0. An existing connection was forcibly closed by the remote host.”
Superseded Software Packages Not Removed Before You Install the Prerequisites
The software packages referred to in this section must be uninstalled before you install the software prerequisites.
If you have any of the following software packages on your development machine, you need to uninstall them before you install the software prerequisites:
- Microsoft HPC Pack 2008 R2 Azure Edition
- Microsoft HPC Pack 2008 R2 Client Components
- Microsoft HPC Pack 2008 R2 MS-MPI Redistributable Pack
- Microsoft HPC Pack 2008 R2 SDK
- Windows Azure SDK v1.5
- Windows Azure AppFabric v1.5
- Windows Azure Tools for Microsoft VS2010 1.5
Network Error when Creating SQL Server Instance
When you deploy “Cloud Numerics,” for the first time, if you receive an error message when the “Cloud Numerics” Deployment Utility attempts to create your SQL Azure database, it is likely your network rules are blocking SSL traffic on TCP port 8443. In order to create the SQL Azure server instance, the “Cloud Numerics” Deployment Utility makes a network call to the SQL Azure Management API over this port.
To allow the deployment tool to complete the cluster deployment sequence, your site’s firewall rules need to enable SSL traffic over port 8443. The location where this modification needs to be made depends on your network route to SQL Azure. It may involve updating firewall rules on your local machine (where you run the deployment utility) and possibly any proxy or gateway servers through which traffic is filtered.
For details regarding requests sent to the SQL Azure Management API, see the following URL:
https://msdn.microsoft.com/en-us/library/windowsazure/gg715281.aspx
Cannot Establish Connection with SQL Server
Depending on your site IT policy, you might need a gateway client in order to set up a connection with the Azure Storage service.
https://www.microsoft.com/en-us/server-cloud/forefront/threat-management-gateway-trial.aspx
If you see the following message, it is typically a indicator that you need to install a gateway client.
Windows Azure Management Service Reports a Conflict
The management service reports a conflict if you attempt to deploy your cluster with a service name that is already in use.
If you see the following message when you try to deploy a “Cloud Numerics” application using the “Cloud Numerics” Deployment Utility (AppConfigure.exe), this can be an indicator that the Windows Azure service name you chose is already in use.
To work around this situation, you can deploy your “Cloud Numerics” cluster with a different Windows Azure service name.
For example, if your first attempt specified the service name “contoso,” you could either choose another name or add a few characters to the end of this name to differentiate it from the service that is already deployed and in use. For example, you could try again using “contosopharmaceuticals.” Your fully qualified domain name for your Windows Azure service would then be contosopharmaceuticals.cloudapp.net.
Comments
- Anonymous
February 14, 2012
Regarding the "Best Practices" for memory usage: If I understand you right, we should enclose all relevant algorithm parts into using blocks:using (IArray<double> A = 0.5 * (B - B.Transpose()) + 0.5 * (B + B.Transpose())) { // do something with A}So A will be disposed as soon as the block ends. Nice! But how prevents Cloud Numerics the creation of temporary arrays for 'B - B.Transpose()' and other binary operators in the example? Are those arrays (if any) somehow disposed in a timely manner also? - Anonymous
February 15, 2012
The reason for adding "using" blocks is that CLR doesn't know how large numeric dense arrays actually are, and might not garbage collect them in timely manner.In the case of temporaries as in the previous comment, they would get disposed when CLR does garbage collection. You could add more "using" blocks if you find that temporaries are using too much memory.