Share via

Set up an auto-scale environment to operationalize your R analytics, with just ONE CLICK

  • Article

Microsoft R Server offers a powerful feature set to help enterprise to deploy their R analytics as web services. Those web services are the productization of the R analytics and can then be integrated with line of business apps. For enterprise, it is crucial to host those web services in a production environment that is secure, scalable, high available and monitorable. Microsoft R Server has provided guidance on how to set up a grid of servers to scale out and achieve high availability. It is, however, a non-trivial task for IT admins to set up such kind of environment.

Since version 9.1, Microsoft R Server enabled an easy way to set up an enterprise grade operationalization environment in Azure with just 1 click deployment, by using Azure ARM Template. After the deployment, with the rich Azure services, enterprise can get the following capabilities:

  • Zero configuration: Spin up a grid of servers to host web services, without manually configuring those servers one by one (the template will do it for you). By default, you will have a high available grid up and running, with
    • 2 web nodes,
    • 2 compute nodes,
    • a web load balancer for the 2 web nodes
    • an Azure SQL DB to store the service payloads
  • Auto-scale: Both the web nodes and compute nodes can automatically scale up and down, based on the CPU load, or the input/output data load or disk load etc.
  • Monitor and alert: You can see the resource running metrics in dashboard, and set up your own rules to send alert notifications of the operation status.

This blog will explain how to do it, with a step by step guidance.

Step 1: Spin up the grid

  1. Go to GitHub: https://github.com/Microsoft/microsoft-r/tree/master/rserver-arm-templates/enterprise-configuration/windows-sql-azure and click the "Deploy to Azure" button. The GitHub page also described the overall architecture of this ARM template.

 

  1. Fill in the required information for the deployment. Here are some important tips:
  • Resource Group: you may want to create a new one. There will be more than 10 resources created. It is easier to manage them in a new group dedicated for this set up.
  • To understand some terminologies:
    -
    "Web Node" and "Compute Node";
  • Web Node Instance Count and Compute Node Instance Count: we suggest you keep the default value. It is the minimal requirement to maintain a high availability set up. In another word, with this set up, if any one of the nodes is down, the whole system can still be up and running.
  • Admin Username: this is used to connect (RDP) to the web nodes.
  • Admin Password: you must follow the local admin password policy defined here: " This local 'admin' password must be 8-16 characters long and contain at least 1 uppercase character(s), 1+ lowercase character(s), 1+ number(s), and 1+ special character(s) . " If you didn't follow this rule, you may face up the "cannot connect to server" error when you use remoteLogin command later.
  • Recommend you "Pin" this resource group to dashboard.

  1. After you accepted the license terms and clicked on the "Purchase" button, it will take ~15 minutes to complete the deployment. This is how easy to set it up!
  2. After the deployment is completed, click the Resource Group you just created from your Azure dashboard, you will see the overview of all the resources.

  1. We will now test if the grid works.
    1. First, you need to get the public IP address. Click and open the "PublicIP" resource, listed as one of the 14 resources in the group. Then copy either the "IP address" or "DNS name" (see the chart below).
    2. From your R IDE with Microsoft R Client V3.3.2 (or later version), type in the following R scripts:

remoteLogin("https://<IP address or DNS name>:12800")
  1. A dialog will pop up to ask you to input the user name and passwork.
    1. Use name is always "admin", *not* the one you entered during deployment.
    2. The password is the one you entered during deployment.
  2. If you see the command line sign changed from ">" to "Remote >", congratulations! The grid works.

  1. If the grid didn't work, please double check if you got the password setup correctly following the rules mentioned above. In most cases, deploy a new one by following the above guidance carefully, you should be able to get it work smoothly.
  2. If you want to let the users using their company domain to access this grid, you can integrate this grid with Azure Active Directory. Please follow this guidance to set up the AAD integration. For this case, you need to use RDP connection to both web nodes, and change their "appsettings.json" files respectively.

Step 2: Auto Scale

After the grid is up and running, you can fine tune how do you want the grid to scale up and down.

  1. Go to the Resource Group Overview page, and click on the "ComputeNodes" resource. In the Navigation pane on the left side, click on the "Scaling". You will see the default "Scale out" and "Scale in" rules.

  1. You can click on those rules and modify them. There are many aspects you can configure. Please explore and choose the one fit your needs.

  1. You can monitor the scale up and down history from the "Run History" tap and set up Notifications in the "Notify" tab at the top of the "Scaling" page.

  1. You can do the configuration, monitor and notify for "WebNodes" the same way.

 

Step 3: Monitor and Notify

It is important for enterprise IT to monitor the resource usage and be notified when abnormal events occur. Azure offers great services to support those requirements.

  1. In the Resource Group àMonitoring section in the left pane, click "Metrics", you can see the key metrics for all the resources.

 

  1. For each resource, and each metrics, you can add rules to notify you, through email.

 

In summary, with this Azure ARM template, you are able to set up an enterprise grade production environment in Azure with just ONE CLICK to host your R-based web services, with auto scale up and down and great monitoring and notification capabilities.

There are more templates available for different platforms and different configurations. please refer to this blog for all the offers.

Useful Resources

Introduction:

Get Started:

Be an expert: