Step 1: Prepare for Your Deployment
Applies To: Microsoft HPC Pack 2012, Microsoft HPC Pack 2012 R2
The first step in the deployment of your HPC cluster is to make important decisions, such as deciding how you will be adding nodes to your cluster, and choosing a network topology for your cluster. The following checklist describes the steps involved in preparing for your deployment.
Checklist: Prepare for your deployment
Task |
Description |
|---|---|
1.1. Review the system requirements |
Review the system requirements to ensure that you have all the necessary hardware and software components to deploy an HPC cluster. |
1.2. Decide if you want to configure your head node for high availability |
Decide if you will need to configure the head node for high availability to enable the cluster to continue running jobs during planned and unplanned downtime. |
1.3. Decide if you want to deploy your cluster with remote databases |
Decide if you want to install the HPC databases on one or more remote servers, instead of installing them on the head node of your cluster. |
1.4. Decide what type of nodes you want to add to your cluster and how many |
Decide if you want to add compute nodes, broker nodes, workstation or unmanaged server nodes, or Windows Azure nodes to your cluster. Also, decide how many nodes to add. |
1.5. Decide how to add on-premises nodes to your cluster |
If you will be adding on-premises nodes to your cluster, decide if you will be adding node from bare metal, from preconfigured nodes, or from an XML file. Decide also if you want to deploy nodes over Internet SCSI (iSCSI). |
1.6. Choose the Active Directory domain for your cluster |
Choose the Active Directory domain to which you will join the head node and the other nodes of your HPC cluster. |
1.7. Choose a domain account for adding nodes |
Choose an existing domain account with enough privileges to add on-premises nodes to the cluster. |
1.8. Choose a network topology for your cluster |
Choose how the nodes in your cluster will be connected and how the cluster will be connected to your enterprise network. |
1.9. Prepare for multicast (optional) |
If you will be deploying nodes from bare metal and want to multicast the operating system image that you will be using during deployment, configure your network switches appropriately. |
1.10. Prepare for the integration of scripted power control tools (optional) |
If you want to use your own power controls tools to start, shut down, and reboot nodes remotely, obtain and test all the necessary components of your power control tools. |
1.1. Review the system requirements
If you have not done so already, review the hardware, software, and related requirements to run HPC Pack at System Requirements for Microsoft HPC Pack 2012 R2 and HPC Pack 2012. Note that HPC Pack has different requirements for different node roles and deployment options. You might want to review the system requirements again, after you have finalized the decisions for your deployment.
1.2. Decide if you want to configure your head node for high availability
If you will need to continue running HPC jobs during a planned or unplanned disruption in services on a head node computer, you can plan to configure the head node for high availability in the context of a Windows Server failover cluster. To do this, you will need to install HPC Pack on two or more head node computers that are configured in a failover cluster, with appropriate hardware, cluster storage, and highly available file services. You can also configure a multisite cluster that can be used for disaster recovery if an interruption occurs at the primary cluster site. Configuring the head node for high availability will require additional planning, hardware, server software, and configuration steps that are beyond the scope of this guide. For general considerations, see 2.3. Configure a Windows Server failover cluster to enable high availability of the head node (optional), later in this guide.
Note
-
You can install HPC Pack 2012 R2 or HPC Pack 2012 on up to 64 head nodes in a single cluster.
-
You can also configure WCF broker nodes in one or more failover clusters to provide high availability of service-oriented architecture (SOA) applications on the cluster.
-
If you are planning to configure cluster nodes for high availability, ensure that you plan and prepare the necessary Windows Server 2012 R2 or Windows Server 2012 failover cluster before you install HPC Pack.
1.3. Decide if you want to deploy your cluster with remote databases
HPC Pack 2012 R2 or HPC Pack 2012 requires and supports Microsoft SQL Server 2012 or Microsoft SQL Server 2008 R2. HPC Pack uses five different SQL Server databases to store cluster management, job scheduling, reporting, diagnostics, and monitoring data. You can install one or more of these five HPC databases on one or more remote servers, instead of installing them on the head node of your cluster. By default, HPC Pack installs SQL Server Express 2012 on the head node and creates the HPC databases on the head node. The advantage of installing the HPC databases on one or more remote servers is that it saves resources on the head node, helping ensure that it can efficiently manage the cluster.
Important
Use of SQL Server 2012 Express on the head node is recommended for proof-of-concept or development clusters, and for smaller production clusters. You should consider installing the HPC databases on one or more remote servers if your cluster will have more than 256 nodes, you plan to configure the head node for high availability in the context of a failover cluster, or your job throughput and reporting requirements could exceed the capabilities of SQL Server 2012 Express.
To install the HPC databases on a remote server, that server must be running the Standard or Enterprise edition of SQL Server 2008 R2 or SQL Server 2012, and configured to work with HPC Pack. Before you install HPC Pack with remote databases, ask the database administrator to run the SetupHpcDatabase.cmd script in the Setup folder or to manually perform or modify the tasks in the script. The script automatically creates the necessary databases and the SQL instance logins and database users for the account that will install HPC Pack and for the machine account for HPC services. For detailed information, see Deploying a Windows HPC Cluster with Remote Databases Step-by-Step Guide.
1.4. Decide what type of nodes you want to add to your cluster and how many
You can add the following types of nodes to your cluster:
Compute nodes. Compute nodes are used for running jobs. This type of node cannot become a different type of node (that is, change roles) without being redeployed.
Broker nodes. Windows Communication Foundation (WCF) broker nodes are used for routing WCF calls from the Service-Oriented Architecture (SOA) clients to the SOA services running on nodes in your cluster. This type of node can change roles to become a compute node without being redeployed.
Workstation nodes and unmanaged server nodes. Workstation nodes and unmanaged server nodes are computers in your organization that can also run jobs, but they are not dedicated cluster resources. They can be scheduled to become available to run jobs at specific times, or can be made available on demand. This type of node cannot change roles.
Windows Azure nodes. If you have a Windows Azure subscription, you can add Windows Azure nodes on demand to increase your cluster capacity when you need it. Like compute nodes, workstation nodes, and unmanaged server nodes, Windows Azure nodes can run jobs. When you add Windows Azure nodes, you also configure a fixed or variable number of proxy nodes in your Windows Azure deployment to facilitate communication between the on-premises head node and the Windows Azure nodes.
For more information about node roles in a Windows HPC cluster, see Understanding Node Roles in Microsoft HPC Pack.
When HPC Pack is installed, depending on the type of node that is being created, different features are installed. These features determine the role that the node will perform in the cluster. In some cases, a node is able to change roles because it has the necessary features to perform a different role. The ability to change roles is an important aspect that you need to consider when deciding the type of nodes that you want to add to your cluster.
Another important decision that you have to make is the number of nodes that you want to add. If you are adding broker nodes, you also need to decide how many compute nodes you will add for each broker node that is available on the cluster. The ratio of broker nodes to compute nodes can affect cluster performance. If you plan to add Windows Azure nodes, you should consider the number of proxy nodes that is optimal for the number of nodes deployed in Windows Azure and the jobs that will run on those nodes. The proxy nodes are required for communication with the on-premises head node and can be a bottleneck for certain cluster sizes and workloads.
Finally, if you want to configure the head node or a broker node in a failover cluster, you will need one additional computer for each failover cluster node that you configure, which might reduce the number of compute nodes that you can add to your cluster.
1.5. Decide how to add on-premises nodes to your cluster
There are three ways to add on-premises nodes to your HPC cluster:
Deploy nodes from bare metal. The operating system and all the necessary HPC cluster features are automatically installed on each node as it is added to the cluster. No manual installation of the operating system or other software is required. Bare metal deployment is only possible for compute nodes and broker nodes.
Add preconfigured nodes. The nodes are already running one of the supported operating systems, and HPC Pack is first manually installed on each node. This is the only way to add workstation nodes and unmanaged server nodes to your cluster. You can also add preconfigured compute nodes and broker nodes.
Import a node XML file. A node XML file contains a list of all the nodes that will be added to the cluster. This XML file can be used to add preconfigured nodes or to deploy nodes from bare metal. For more information about node XML files, see Appendix 2: Creating a Node XML File, later in this guide
The following are considerations when choosing how to add nodes to your HPC cluster:
When deploying nodes from bare metal, HPC Pack automatically generates computer names for your nodes. During the configuration process for the head node of the cluster, you will be required to specify the naming convention to use when automatically generating computer names for the new nodes.
Nodes are assigned their computer name in the order that they are deployed.
If you want to add nodes from bare metal and assign computer names in a different way, you can use a node XML file. For more information about node XML files, see Appendix 2: Creating a Node XML File in the Design and Deployment Guide.
Deploy on-premises nodes over iSCSI
You can centralize the storage of your HPC cluster by using a network-attached storage array. A networked-attached storage array is a computer, storage system, or appliance that provides storage resources over a network connection.
By using a storage array, the nodes in your cluster will not require a local hard disk drive to serve as a system disk, and instead use the storage resources on the storage array to boot the operating system over the network, using an iSCSI connection.
Nodes that are deployed over iSCSI are deployed from bare metal.
Important
To deploy nodes over iSCSI in a cluster that is running HPC Pack 2012, ensure that the cluster is updated with at least Service Pack 1.
To have an iSCSI deployment, you will need the following:
One or more network-attached storage arrays.
A network connection between the nodes in your cluster and the storage arrays.
An iSCSI provider for the storage arrays, installed on the head node. An iSCSI provider that is compatible with HPC Pack is available from the Microsoft Download Center.
Note
For detailed information about iSCSI deployment and step-by-step procedures for deploying iSCSI boot nodes, see the content on deploying iSCSI boot nodes in a Windows HPC cluster.
1.6. Choose the Active Directory domain for your cluster
The nodes in your HPC cluster must be members of an Active Directory domain. Before deploying your cluster, you must choose the Active Directory domain that you will use for your HPC cluster.
Depending on the Active Directory environment in your organization, it may be helpful to configure a separate organizational unit (OU) for the computers that will be members of the HPC cluster. With a separate OU, if necessary, different policies and settings can be applied to the cluster nodes than to the other computers in your organization.
If you do not have an Active Directory domain to which you can join your cluster, or if you prefer not to join an existing domain, you can install the Active Directory Domain Services role on a computer that is running Windows Server 2012 R2 or Windows Server 2012, and then configure a domain controller on that computer. For more information about installing the Active Directory Domain Services role, see Deploying Active Directory Domain Services (AD DS) in Your Enterprise.
The following are additional considerations about the Active Directory domain for your cluster:
Because of potential administrative difficulties and to ensure cluster performance, we do not recommend using the head node as a domain controller unless isolation is required (for example, for test purposes) or no other option exists.
If you choose to install and configure an Active Directory domain controller on the head node, consult with your network administrator about the correct way to isolate the new Active Directory domain from the enterprise network, or how to join the new domain to an existing Active Directory forest.
If you plan to add workstation nodes or unmanaged server nodes to the HPC cluster, those computers can be joined to any Active Directory domain that has an established trust relationship with the domain to which the head node is joined.
1.7. Choose a domain account for adding nodes
To install HPC Pack on the head node, you must be logged on with a domain user account that is a member of the Administrators group on the head node computer. Additionally, during the configuration process of your HPC head node after the installation of HPC Pack, you must provide credentials for a domain user account that will be used for adding on-premises nodes and for system configuration of those nodes. You must choose an existing account or create a new account before starting your cluster deployment.
The following is a list of details to take into consideration when choosing the user account for adding nodes:
The user account that you choose must be a domain account with enough privileges to create Active Directory computer accounts for the nodes and to join the nodes to the domain.
If the policies of your organization restrict you from using a domain account that can add new computers to the domain, you will need to ask your domain administrator to pre-create the computer objects for you in Active Directory Domain Services before you deploy your nodes. For more information, see Deploy Nodes with Pre-created Computer Objects in Active Directory.
If part of your deployment requires access to resources on the enterprise network, the user account must have the necessary permissions to access those resources—for example, installation files that are available on a network server.
If you want to restart nodes remotely by using HPC Cluster Manager, the account must be a member of the local Administrators group on the head node. This requirement is only necessary if you do not have scripted power control tools that you can use to remotely restart the nodes.
1.8. Choose a network topology for your cluster
HPC Pack supports five cluster topologies. These topologies are distinguished by how the nodes in the cluster are connected to each other and to the enterprise network. The five supported cluster topologies are:
Topology 1: Compute nodes isolated on a private network
Topology 2: All nodes on enterprise and private networks
Topology 3: Compute nodes isolated on private and application networks
Topology 4: All nodes on enterprise, private, and application networks
Topology 5: All nodes on an enterprise network
For more information about each network topology and each HPC cluster network, see Appendix 1: HPC Cluster Networking, later in this guide.
When you are choosing a network topology, you must take into consideration your existing network infrastructure and the type of nodes that you will be adding to your cluster:
Decide which network in the topology that you have chosen will serve as the enterprise network, the private network, and the application network.
Do not have the network adapter that is connected to the enterprise network on the head node in automatic configuration (that is, the IP address for that adapter does not start with: 169.254). That adapter must have a valid IP address, dynamically or manually assigned (static).
If you choose a topology that includes a private network, and you are planning to add nodes to your cluster from bare metal, do the following:
Ensure that there are no Pre-Boot Execution Environment (PXE) servers on the private network.
If you want to use an existing DHCP server for your private network, ensure that it is configured to recognize the head node as the PXE server in the network.
If you want to enable DHCP server on your head node for the private or application networks and there are other DHCP servers connected to those networks, you must disable those DHCP servers.
If you have an existing Domain Name System (DNS) server connected to the same network as the nodes in your cluster, no action is necessary, but the nodes will be automatically deregistered from that DNS server.
Contact your system administrator to determine if Internet Protocol security (IPsec) is enforced on your domain through Group Policy. If IPsec is enforced on your domain through Group Policy, you may experience issues during deployment. A workaround is to make your head node an IPsec boundary server so that the other nodes in your cluster can communicate with the head node during PXE boot.
If you want to add workstation nodes or unmanaged server nodes to your cluster, topology 5 (all nodes on an enterprise network) is the recommended topology, but other topologies are supported. If you want to add workstation nodes on other topologies, see the content on Adding Workstation Nodes to a Windows HPC cluster.
If you want to add broker nodes to your cluster, they must be connected to the network where the clients that are starting SOA sessions are connected (usually the enterprise network) and to the network where the nodes that are running the SOA services are connected (if different from the network where the clients are connected).
If you want to add Windows Azure nodes to your cluster, your HPC cluster can be configured in any cluster network topology that is supported by HPC Pack. The head node and any client computer that is used to manage the cluster and that needs a connection to Windows Azure must be able to connect over the Internet to Windows Azure services.
1.9. Prepare for multicast (optional)
If you will be deploying on-premises nodes from bare metal and want to multicast the operating system image that you will be using during deployment, we recommend that you prepare for multicast by:
Enabling Internet Group Management Protocol (IGMP) snooping on your network switches, if this feature is available. This will help to reduce multicast traffic.
Disabling Spanning Tree Protocol (STP) on your network switches, if this feature is enabled.
Note
For more information about these settings, contact your network administrator or your networking hardware vendor.
1.10. Prepare for the integration of scripted power control tools (optional)
The cluster administration console (HPC Cluster Manager) includes actions to start, shut down, and reboot nodes remotely. These actions are linked to a script file (CcpPower.cmd) that performs these power control operations by using operating system commands. You can replace the default operating system commands in that script file with your own power control scripts, such as Intelligent Platform Management Interface (IPMI) scripts provided by your vendor of cluster solutions.
In preparation for this integration, you must obtain all the necessary scripts, .dll files, and other components of your power control tools. After you have obtained all the necessary components, test them independently and ensure that they work as intended on the computers that you will be deploying as nodes in your cluster.
For information about modifying CcpPower.cmd to integrate your own scripted power control tools, see Appendix 5: Scripted Power Control Tools, later in this guide.