New-AzureHDInsightCluster

Reference

Module:: Azure

Creates an HDInsight cluster.

Note

The cmdlets referenced in this documentation are for managing legacy Azure resources that use Azure Service Manager (ASM) APIs. This legacy PowerShell module isn't recommended when creating new resources since ASM is scheduled for retirement. For more information, see Azure Service Manager retirement.

The Az PowerShell module is the recommended PowerShell module for managing Azure Resource Manager (ARM) resources with PowerShell.

Syntax

New-AzureHDInsightCluster
   [-Certificate <X509Certificate2>]
   [-HostedService <String>]
   -Config <AzureHDInsightConfig>
   -Credential <PSCredential>
   [-EndPoint <Uri>]
   [-IgnoreSslErrors <Boolean>]
   -Location <String>
   -Name <String>
   [-Subscription <String>]
   [-Version <String>]
   [-OSType <OSType>]
   [-SshCredential <PSCredential>]
   [-SshPublicKey <String>]
   [-RdpCredential <PSCredential>]
   [-RdpAccessExpiry <DateTime>]
   [-Profile <AzureSMProfile>]
   [<CommonParameters>]

New-AzureHDInsightCluster
   [-Certificate <X509Certificate2>]
   [-HostedService <String>]
   -ClusterSizeInNodes <Int32>
   -Credential <PSCredential>
   -DefaultStorageAccountKey <String>
   -DefaultStorageAccountName <String>
   -DefaultStorageContainerName <String>
   [-EndPoint <Uri>]
   [-IgnoreSslErrors <Boolean>]
   -Location <String>
   -Name <String>
   [-Subscription <String>]
   [-Version <String>]
   [-HeadNodeVMSize <String>]
   [-ClusterType <ClusterType>]
   [-VirtualNetworkId <String>]
   [-SubnetName <String>]
   [-DataNodeVMSize <String>]
   [-ZookeeperNodeVMSize <String>]
   [-OSType <OSType>]
   [-SshCredential <PSCredential>]
   [-SshPublicKey <String>]
   [-RdpCredential <PSCredential>]
   [-RdpAccessExpiry <DateTime>]
   [-Profile <AzureSMProfile>]
   [<CommonParameters>]

Description

This version of Azure PowerShell HDInsight is deprecated. These cmdlets will be removed by January 1, 2017. Please use the newer version of Azure PowerShell HDInsight.

For information about how to use the new HDInsight to create a cluster, see Create Linux-based clusters in HDInsight using Azure PowerShell (https://azure.microsoft.com/en-us/documentation/articles/hdinsight-hadoop-create-linux-clusters-azure-powershell/). For information about how to submit jobs by using Azure PowerShell and other approaches, see Submit Hadoop jobs in HDInsight (https://azure.microsoft.com/en-us/documentation/articles/hdinsight-submit-hadoop-jobs-programmatically/). For reference information about Azure PowerShell HDInsight, see Azure HDInsight Cmdlets.

The New-AzureHDInsightCluster cmdlet creates an Azure HDInsight cluster by using the specified parameters or by using a configuration object that is created by using the New-AzureHDInsightClusterConfig cmdlet.

Examples

Example 1: Create an HDInsight cluster

PS C:\>$SubId = (Get-AzureSubscription -Current).SubscriptionId
PS C:\> $Key1 = Get-AzureStorageKey -StorageAccountName "MyBlobStorage" | %{ $_.Primary }
PS C:\> $Key2 = Get-AzureStorageKey -StorageAccountName "MySecondBlobStorage" | %{ $_.Primary }
PS C:\> $Creds = Get-Credential
PS C:\> $OozieCreds = Get-Credential
PS C:\> $HiveCreds = Get-Credential
PS C:\> New-AzureHDInsightClusterConfig -ClusterSizeInNodes 4
    | Set-AzureHDInsightDefaultStorage -StorageAccountName "MyBlobStorage.blob.core.windows.net" -StorageAccountKey $Key1 -StorageContainerName "MyContainer"
    | Add-AzureHDInsightStorage -StorageAccountName "MySecondBlobStorage.blob.core.windows.net" -StorageAccountKey $Key2
    | Add-AzureHDInsightMetastore -SqlAzureServerName "MySqlServer.database.windows.net" -DatabaseName "MyOozieDatabaseName" -Credential $OozieCreds -MetastoreType OozieMetastore
    | Add-AzureHDInsightMetastore -SqlAzureServerName "MySqlServer.database.windows.net" -DatabaseName "MyHiveDatabaseName" -Credential $HiveCreds -MetastoreType HiveMetastore
    | New-AzureHDInsightCluster -Subscription $SubId -Credential $Creds

This example creates an HDInsight cluster for the current subscription.

The first command uses the Get-AzureSubscription cmdlet to get the current subscription ID, and then stores it in the $SubId variable.

The second and third commands use the Get-AzureStorageKey cmdlet to get the primary storage keys for MyBlobStorage and MySecondBlobStorage, and then store the keys in the $Key1 and $Key2 variables, respectively.

The fourth, fifth, and sixth commands use the Get-Credential cmdlet to get credentials for the current subscription and for Oozie and Hive, and then store the credentials in variables.

The final command performs a sequence of operations by using these cmdlets:

New-AzureHDInsightClusterConfig to create an HDInsight cluster configuration.
Set-AzureHDInsightDefaultStorage to set the default storage account for the configuration to MyBlobStorage.blob.core.windows.net.
Add-AzureHDInsightStorage to add a second storage account named MySecondBlobStorage.blob.core.windows.net to the configuration.
Add-AzureHDInsightMetastore to add a metastore for Oozie and a metastore for Hive to the configuration.
New-AzureHDInsightCluster to create an HDInsight cluster with the new configuration.

Parameters

-Certificate

Specifies the management certificate for an Azure subscription.

Type:	X509Certificate2
Aliases:	Cert
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-ClusterSizeInNodes

Specifies the number of data nodes to create for a cluster.

Type:	Int32
Aliases:	Nodes, Size
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	False
Accept wildcard characters:	False

-ClusterType

Specifies the type of cluster to create.

Type:	ClusterType
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-Config

Specifies a configuration object that is created by using the New-AzureHDInsightClusterConfig cmdlet.

Type:	AzureHDInsightConfig
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	True
Accept wildcard characters:	False

-Credential

Specifies the user credentials for HDInsight to use for the default account that is used to remotely access a Hadoop cluster. These credentials are distinct from the subscription credentials of the user.

Type:	PSCredential
Aliases:	Cred
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	False
Accept wildcard characters:	False

-DataNodeVMSize

Specifies the size of the virtual machine for the data node.

Type:	String
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-DefaultStorageAccountKey

Specifies the account key for the default storage account that the HDInsight cluster uses.

Type:	String
Aliases:	StorageKey
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	False
Accept wildcard characters:	False

-DefaultStorageAccountName

Specifies the name of the default storage account that the HDInsight cluster uses.

Type:	String
Aliases:	StorageAccount
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	False
Accept wildcard characters:	False

-DefaultStorageContainerName

Specifies the name of the default container in the default Azure storage account that an HDInsight cluster uses.

Type:	String
Aliases:	StorageContainer
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	False
Accept wildcard characters:	False

-EndPoint

Specifies the endpoint to use to connect to Azure. If you do not specify this parameter, this cmdlet uses the default endpoint.

Type:	Uri
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-HeadNodeVMSize

Specifies the size of the virtual machine for the head node.

Type:	String
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-HostedService

Specifies the namespace of an HDInsight service. If you do not specify this parameter, this cmdlet uses the default namespace.

Type:	String
Aliases:	CloudServiceName
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-IgnoreSslErrors

Indicates whether Secure Sockets Layer (SSL) errors are ignored.

Type:	Boolean
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-Location

Specifies the region in which to create an HDInsight cluster.

Type:	String
Aliases:	Loc
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	False
Accept wildcard characters:	False

-Name

Specifies the name of the Azure HDInsight cluster to create.

Type:	String
Aliases:	ClusterName, DnsName
Position:	Named
Default value:	None
Required:	True
Accept pipeline input:	False
Accept wildcard characters:	False

-OSType

Specifies the operating system for a cluster.

Type:	OSType
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-Profile

Specifies the Azure profile from which this cmdlet reads. If you do not specify a profile, this cmdlet reads from the local default profile.

Type:	AzureSMProfile
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-RdpAccessExpiry

Specifies the expiration, as a DateTime object, for Remote Desktop Protocol (RDP) access to a cluster.

Type:	DateTime
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-RdpCredential

Specifies the credentials for RDP access to a cluster.

Type:	PSCredential
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-SshCredential

Specifies the Secure Shell (SSH) user name and password for the HDInsight cluster. This parameter is valid only for Linux clusters.

Type:	PSCredential
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-SshPublicKey

Specifies the SSH public key for an HDInsight cluster. This parameter is valid only for Linux clusters.

Type:	String
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-SubnetName

Specifies the name of a subnet.

Type:	String
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-Subscription

Specifies the Azure subscription in which to create an HDInsight cluster.

Type:	String
Aliases:	Sub
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-Version

Specifies the HDInsight cluster version to create.

Type:	String
Aliases:	Ver
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-VirtualNetworkId

Specifies the ID of the virtual network into which to provision the cluster.

Type:	String
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

-ZookeeperNodeVMSize

Specifies the size of the virtual machine for the ZooKeeper node. This parameter is valid only for HBase or Storm clusters.

Type:	String
Position:	Named
Default value:	None
Required:	False
Accept pipeline input:	False
Accept wildcard characters:	False

Share via

New-AzureHDInsightCluster

Syntax

Description

Examples

Example 1: Create an HDInsight cluster

Parameters

-Certificate

-ClusterSizeInNodes

-ClusterType

-Config

-Credential

-DataNodeVMSize

-DefaultStorageAccountKey

-DefaultStorageAccountName

-DefaultStorageContainerName

-EndPoint

-HeadNodeVMSize

-HostedService

-IgnoreSslErrors

-Location

-Name

-OSType

-Profile

-RdpAccessExpiry

-RdpCredential

-SshCredential

-SshPublicKey

-SubnetName

-Subscription

-Version

-VirtualNetworkId

-ZookeeperNodeVMSize

Additional resources

Share via

New-AzureHDInsightCluster

Syntax

Description

Examples

Example 1: Create an HDInsight cluster

Parameters

-Certificate

-ClusterSizeInNodes

-ClusterType

-Config

-Credential

-DataNodeVMSize

-DefaultStorageAccountKey

-DefaultStorageAccountName

-DefaultStorageContainerName

-EndPoint

-HeadNodeVMSize

-HostedService

-IgnoreSslErrors

-Location

-Name

-OSType

-Profile

-RdpAccessExpiry

-RdpCredential

-SshCredential

-SshPublicKey

-SubnetName

-Subscription

-Version

-VirtualNetworkId

-ZookeeperNodeVMSize

Related Links

Additional resources