Bicep resource definition
The clusters resource type can be deployed with operations that target:
For a list of changed properties in each API version, see change log.
To create a Microsoft.HDInsight/clusters resource, add the following Bicep to your template.
resource symbolicname 'Microsoft.HDInsight/clusters@2024-08-01-preview' = {
identity: {
type: 'string'
userAssignedIdentities: {
{customized property}: {
tenantId: 'string'
}
}
}
location: 'string'
name: 'string'
properties: {
clusterDefinition: {
blueprint: 'string'
componentVersion: {
{customized property}: 'string'
}
configurations: any(Azure.Bicep.Types.Concrete.AnyType)
kind: 'string'
}
clusterVersion: 'string'
computeIsolationProperties: {
enableComputeIsolation: bool
hostSku: 'string'
}
computeProfile: {
roles: [
{
autoscale: {
capacity: {
maxInstanceCount: int
minInstanceCount: int
}
recurrence: {
schedule: [
{
days: [
'string'
]
timeAndCapacity: {
maxInstanceCount: int
minInstanceCount: int
time: 'string'
}
}
]
timeZone: 'string'
}
}
dataDisksGroups: [
{
disksPerNode: int
}
]
encryptDataDisks: bool
hardwareProfile: {
vmSize: 'string'
}
minInstanceCount: int
name: 'string'
osProfile: {
linuxOperatingSystemProfile: {
password: 'string'
sshProfile: {
publicKeys: [
{
certificateData: 'string'
}
]
}
username: 'string'
}
}
scriptActions: [
{
name: 'string'
parameters: 'string'
uri: 'string'
}
]
targetInstanceCount: int
virtualNetworkProfile: {
id: 'string'
subnet: 'string'
}
VMGroupName: 'string'
}
]
}
diskEncryptionProperties: {
encryptionAlgorithm: 'string'
encryptionAtHost: bool
keyName: 'string'
keyVersion: 'string'
msiResourceId: 'string'
vaultUri: 'string'
}
encryptionInTransitProperties: {
isEncryptionInTransitEnabled: bool
}
kafkaRestProperties: {
clientGroupInfo: {
groupId: 'string'
groupName: 'string'
}
configurationOverride: {
{customized property}: 'string'
}
}
minSupportedTlsVersion: 'string'
networkProperties: {
outboundDependenciesManagedType: 'string'
privateLink: 'string'
publicIpTag: {
ipTagType: 'string'
tag: 'string'
}
resourceProviderConnection: 'string'
}
osType: 'string'
privateLinkConfigurations: [
{
name: 'string'
properties: {
groupId: 'string'
ipConfigurations: [
{
name: 'string'
properties: {
primary: bool
privateIPAddress: 'string'
privateIPAllocationMethod: 'string'
subnet: {
id: 'string'
}
}
}
]
}
}
]
securityProfile: {
aaddsResourceId: 'string'
clusterUsersGroupDNs: [
'string'
]
directoryType: 'string'
domain: 'string'
domainUsername: 'string'
domainUserPassword: 'string'
ldapsUrls: [
'string'
]
msiResourceId: 'string'
organizationalUnitDN: 'string'
}
storageProfile: {
storageaccounts: [
{
container: 'string'
enableSecureChannel: bool
fileshare: 'string'
fileSystem: 'string'
isDefault: bool
key: 'string'
msiResourceId: 'string'
name: 'string'
resourceId: 'string'
saskey: 'string'
}
]
}
tier: 'string'
}
tags: {
{customized property}: 'string'
}
zones: [
'string'
]
}
Property values
Autoscale
AutoscaleCapacity
Name |
Description |
Value |
maxInstanceCount |
The maximum instance count of the cluster |
int |
minInstanceCount |
The minimum instance count of the cluster |
int |
AutoscaleRecurrence
Name |
Description |
Value |
schedule |
Array of schedule-based autoscale rules |
AutoscaleSchedule[] |
timeZone |
The time zone for the autoscale schedule times |
string |
AutoscaleSchedule
Name |
Description |
Value |
days |
Days of the week for a schedule-based autoscale rule |
String array containing any of: 'Friday' 'Monday' 'Saturday' 'Sunday' 'Thursday' 'Tuesday' 'Wednesday' |
timeAndCapacity |
Time and capacity for a schedule-based autoscale rule |
AutoscaleTimeAndCapacity |
AutoscaleTimeAndCapacity
Name |
Description |
Value |
maxInstanceCount |
The maximum instance count of the cluster |
int |
minInstanceCount |
The minimum instance count of the cluster |
int |
time |
24-hour time in the form xx:xx |
string |
ClientGroupInfo
Name |
Description |
Value |
groupId |
The AAD security group id. |
string |
groupName |
The AAD security group name. |
string |
ClusterCreatePropertiesOrClusterGetProperties
ClusterDefinition
Name |
Description |
Value |
blueprint |
The link to the blueprint. |
string |
componentVersion |
The versions of different services in the cluster. |
ClusterDefinitionComponentVersion |
configurations |
The cluster configurations. |
any |
kind |
The type of cluster. |
string |
ClusterDefinitionComponentVersion
ClusterIdentity
Name |
Description |
Value |
type |
The type of identity used for the cluster. The type 'SystemAssigned, UserAssigned' includes both an implicitly created identity and a set of user assigned identities. |
'None' 'SystemAssigned' 'SystemAssigned, UserAssigned' 'UserAssigned' |
userAssignedIdentities |
The list of user identities associated with the cluster. The user identity dictionary key references will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}'. |
ClusterIdentityUserAssignedIdentities |
ClusterIdentityUserAssignedIdentities
ComputeIsolationProperties
Name |
Description |
Value |
enableComputeIsolation |
The flag indicates whether enable compute isolation or not. |
bool |
hostSku |
The host sku. |
string |
ComputeProfile
Name |
Description |
Value |
roles |
The list of roles in the cluster. |
Role[] |
DataDisksGroups
Name |
Description |
Value |
disksPerNode |
The number of disks per node. |
int |
DiskEncryptionProperties
Name |
Description |
Value |
encryptionAlgorithm |
Algorithm identifier for encryption, default RSA-OAEP. |
'RSA-OAEP' 'RSA-OAEP-256' 'RSA1_5' |
encryptionAtHost |
Indicates whether or not resource disk encryption is enabled. |
bool |
keyName |
Key name that is used for enabling disk encryption. |
string |
keyVersion |
Specific key version that is used for enabling disk encryption. |
string |
msiResourceId |
Resource ID of Managed Identity that is used to access the key vault. |
string |
vaultUri |
Base key vault URI where the customers key is located eg. https://myvault.vault.azure.net |
string |
EncryptionInTransitProperties
Name |
Description |
Value |
isEncryptionInTransitEnabled |
Indicates whether or not inter cluster node communication is encrypted in transit. |
bool |
HardwareProfile
Name |
Description |
Value |
vmSize |
The size of the VM |
string |
IPConfiguration
Name |
Description |
Value |
name |
The name of private link IP configuration. |
string (required) |
properties |
The private link ip configuration properties. |
IPConfigurationProperties |
IPConfigurationProperties
Name |
Description |
Value |
primary |
Indicates whether this IP configuration is primary for the corresponding NIC. |
bool |
privateIPAddress |
The IP address. |
string |
privateIPAllocationMethod |
The method that private IP address is allocated. |
'dynamic' 'static' |
subnet |
The subnet resource id. |
ResourceId |
IpTag
Name |
Description |
Value |
ipTagType |
Gets or sets the ipTag type: Example FirstPartyUsage. |
string (required) |
tag |
Gets or sets value of the IpTag associated with the public IP. Example HDInsight, SQL, Storage etc |
string (required) |
KafkaRestProperties
KafkaRestPropertiesConfigurationOverride
LinuxOperatingSystemProfile
Name |
Description |
Value |
password |
The password. |
string |
sshProfile |
The SSH profile. |
SshProfile |
username |
The username. |
string |
Microsoft.HDInsight/clusters
NetworkProperties
Name |
Description |
Value |
outboundDependenciesManagedType |
A value to describe how the outbound dependencies of a HDInsight cluster are managed. 'Managed' means that the outbound dependencies are managed by the HDInsight service. 'External' means that the outbound dependencies are managed by a customer specific solution. |
'External' 'Managed' |
privateLink |
Indicates whether or not private link is enabled. |
'Disabled' 'Enabled' |
publicIpTag |
Gets or sets the IP tag for the public IPs created along with the HDInsight Clusters. |
IpTag |
resourceProviderConnection |
The direction for the resource provider connection. |
'Inbound' 'Outbound' |
OsProfile
PrivateLinkConfiguration
Name |
Description |
Value |
name |
The name of private link configuration. |
string (required) |
properties |
The private link configuration properties. |
PrivateLinkConfigurationProperties (required) |
PrivateLinkConfigurationProperties
Name |
Description |
Value |
groupId |
The HDInsight private linkable sub-resource name to apply the private link configuration to. For example, 'headnode', 'gateway', 'edgenode'. |
string (required) |
ipConfigurations |
The IP configurations for the private link service. |
IPConfiguration[] (required) |
ResourceId
Name |
Description |
Value |
id |
The azure resource id. |
string |
Role
Name |
Description |
Value |
autoscale |
The autoscale configurations. |
Autoscale |
dataDisksGroups |
The data disks groups for the role. |
DataDisksGroups[] |
encryptDataDisks |
Indicates whether encrypt the data disks. |
bool |
hardwareProfile |
The hardware profile. |
HardwareProfile |
minInstanceCount |
The minimum instance count of the cluster. |
int |
name |
The name of the role. |
string |
osProfile |
The operating system profile. |
OsProfile |
scriptActions |
The list of script actions on the role. |
ScriptAction[] |
targetInstanceCount |
The instance count of the cluster. |
int |
virtualNetworkProfile |
The virtual network profile. |
VirtualNetworkProfile |
VMGroupName |
The name of the virtual machine group. |
string |
ScriptAction
Name |
Description |
Value |
name |
The name of the script action. |
string (required) |
parameters |
The parameters for the script provided. |
string (required) |
uri |
The URI to the script. |
string (required) |
SecurityProfile
Name |
Description |
Value |
aaddsResourceId |
The resource ID of the user's Azure Active Directory Domain Service. |
string |
clusterUsersGroupDNs |
Optional. The Distinguished Names for cluster user groups |
string[] |
directoryType |
The directory type. |
'ActiveDirectory' |
domain |
The organization's active directory domain. |
string |
domainUsername |
The domain user account that will have admin privileges on the cluster. |
string |
domainUserPassword |
The domain admin password. |
string |
ldapsUrls |
The LDAPS protocol URLs to communicate with the Active Directory. |
string[] |
msiResourceId |
User assigned identity that has permissions to read and create cluster-related artifacts in the user's AADDS. |
string |
organizationalUnitDN |
The organizational unit within the Active Directory to place the cluster and service accounts. |
string |
SshProfile
Name |
Description |
Value |
publicKeys |
The list of SSH public keys. |
SshPublicKey[] |
SshPublicKey
Name |
Description |
Value |
certificateData |
The certificate for SSH. |
string |
StorageAccount
Name |
Description |
Value |
container |
The container in the storage account, only to be specified for WASB storage accounts. |
string |
enableSecureChannel |
Enable secure channel or not, it's an optional field. Default value is false when cluster version < 5.1 and true when cluster version >= 5.1 , |
bool |
fileshare |
The file share name. |
string |
fileSystem |
The filesystem, only to be specified for Azure Data Lake Storage Gen 2. |
string |
isDefault |
Whether or not the storage account is the default storage account. |
bool |
key |
The storage account access key. |
string |
msiResourceId |
The managed identity (MSI) that is allowed to access the storage account, only to be specified for Azure Data Lake Storage Gen 2. |
string |
name |
The name of the storage account. |
string |
resourceId |
The resource ID of storage account, only to be specified for Azure Data Lake Storage Gen 2. |
string |
saskey |
The shared access signature key. |
string |
StorageProfile
Name |
Description |
Value |
storageaccounts |
The list of storage accounts in the cluster. |
StorageAccount[] |
UserAssignedIdentity
Name |
Description |
Value |
tenantId |
The tenant id of user assigned identity. |
string |
VirtualNetworkProfile
Name |
Description |
Value |
id |
The ID of the virtual network. |
string |
subnet |
The name of the subnet. |
string |
Quickstart samples
The following quickstart samples deploy this resource type.
Bicep File |
Description |
Deploy a Linux-based HBase cluster in HDInsight |
This template allows you to create a Linux-based HBase cluster in Azure HDInsight. |
Deploy a Spark cluster in Azure HDInsight |
This template allows you to create a Spark cluster in Azure HDInsight. |
Deploy an Interative Hive cluster in HDInsight |
This template allows you to create an Interative Hive (LLAP) cluster in HDInsight and the dependent Azure Storage account. The SSH authentication method for the cluster is username and password. For a template using SSH public key authentication, see /samples/azure/azure-quickstart-templates/hdinsight-linux-ssh-publickey |
Deploy HDInsight cluster with Storage and SSH password |
This template allows you to create a Linux-based Hadoop cluster in HDInsight and the dependent Azure Storage account. The SSH authentication method for the cluster is username and password. For a template using SSH public key authentication, see /samples/azure/azure-quickstart-templates/hdinsight-linux-ssh-publickey/ |
Deploy Kafka on HDInsight in a virtual network |
This template allows you to create an Azure Virtual Network and a Kafka on HDInsight cluster in the virtual network. The SSH authentication method for the cluster is username and password. For a template using SSH public key authentication, see /samples/azure/azure-quickstart-templates/hdinsight-linux-ssh-publickey/ |
ARM template resource definition
The clusters resource type can be deployed with operations that target:
For a list of changed properties in each API version, see change log.
To create a Microsoft.HDInsight/clusters resource, add the following JSON to your template.
{
"type": "Microsoft.HDInsight/clusters",
"apiVersion": "2024-08-01-preview",
"name": "string",
"identity": {
"type": "string",
"userAssignedIdentities": {
"{customized property}": {
"tenantId": "string"
}
}
},
"location": "string",
"properties": {
"clusterDefinition": {
"blueprint": "string",
"componentVersion": {
"{customized property}": "string"
},
"configurations": {},
"kind": "string"
},
"clusterVersion": "string",
"computeIsolationProperties": {
"enableComputeIsolation": "bool",
"hostSku": "string"
},
"computeProfile": {
"roles": [
{
"autoscale": {
"capacity": {
"maxInstanceCount": "int",
"minInstanceCount": "int"
},
"recurrence": {
"schedule": [
{
"days": [ "string" ],
"timeAndCapacity": {
"maxInstanceCount": "int",
"minInstanceCount": "int",
"time": "string"
}
}
],
"timeZone": "string"
}
},
"dataDisksGroups": [
{
"disksPerNode": "int"
}
],
"encryptDataDisks": "bool",
"hardwareProfile": {
"vmSize": "string"
},
"minInstanceCount": "int",
"name": "string",
"osProfile": {
"linuxOperatingSystemProfile": {
"password": "string",
"sshProfile": {
"publicKeys": [
{
"certificateData": "string"
}
]
},
"username": "string"
}
},
"scriptActions": [
{
"name": "string",
"parameters": "string",
"uri": "string"
}
],
"targetInstanceCount": "int",
"virtualNetworkProfile": {
"id": "string",
"subnet": "string"
},
"VMGroupName": "string"
}
]
},
"diskEncryptionProperties": {
"encryptionAlgorithm": "string",
"encryptionAtHost": "bool",
"keyName": "string",
"keyVersion": "string",
"msiResourceId": "string",
"vaultUri": "string"
},
"encryptionInTransitProperties": {
"isEncryptionInTransitEnabled": "bool"
},
"kafkaRestProperties": {
"clientGroupInfo": {
"groupId": "string",
"groupName": "string"
},
"configurationOverride": {
"{customized property}": "string"
}
},
"minSupportedTlsVersion": "string",
"networkProperties": {
"outboundDependenciesManagedType": "string",
"privateLink": "string",
"publicIpTag": {
"ipTagType": "string",
"tag": "string"
},
"resourceProviderConnection": "string"
},
"osType": "string",
"privateLinkConfigurations": [
{
"name": "string",
"properties": {
"groupId": "string",
"ipConfigurations": [
{
"name": "string",
"properties": {
"primary": "bool",
"privateIPAddress": "string",
"privateIPAllocationMethod": "string",
"subnet": {
"id": "string"
}
}
}
]
}
}
],
"securityProfile": {
"aaddsResourceId": "string",
"clusterUsersGroupDNs": [ "string" ],
"directoryType": "string",
"domain": "string",
"domainUsername": "string",
"domainUserPassword": "string",
"ldapsUrls": [ "string" ],
"msiResourceId": "string",
"organizationalUnitDN": "string"
},
"storageProfile": {
"storageaccounts": [
{
"container": "string",
"enableSecureChannel": "bool",
"fileshare": "string",
"fileSystem": "string",
"isDefault": "bool",
"key": "string",
"msiResourceId": "string",
"name": "string",
"resourceId": "string",
"saskey": "string"
}
]
},
"tier": "string"
},
"tags": {
"{customized property}": "string"
},
"zones": [ "string" ]
}
Property values
Autoscale
AutoscaleCapacity
Name |
Description |
Value |
maxInstanceCount |
The maximum instance count of the cluster |
int |
minInstanceCount |
The minimum instance count of the cluster |
int |
AutoscaleRecurrence
Name |
Description |
Value |
schedule |
Array of schedule-based autoscale rules |
AutoscaleSchedule[] |
timeZone |
The time zone for the autoscale schedule times |
string |
AutoscaleSchedule
Name |
Description |
Value |
days |
Days of the week for a schedule-based autoscale rule |
String array containing any of: 'Friday' 'Monday' 'Saturday' 'Sunday' 'Thursday' 'Tuesday' 'Wednesday' |
timeAndCapacity |
Time and capacity for a schedule-based autoscale rule |
AutoscaleTimeAndCapacity |
AutoscaleTimeAndCapacity
Name |
Description |
Value |
maxInstanceCount |
The maximum instance count of the cluster |
int |
minInstanceCount |
The minimum instance count of the cluster |
int |
time |
24-hour time in the form xx:xx |
string |
ClientGroupInfo
Name |
Description |
Value |
groupId |
The AAD security group id. |
string |
groupName |
The AAD security group name. |
string |
ClusterCreatePropertiesOrClusterGetProperties
ClusterDefinition
Name |
Description |
Value |
blueprint |
The link to the blueprint. |
string |
componentVersion |
The versions of different services in the cluster. |
ClusterDefinitionComponentVersion |
configurations |
The cluster configurations. |
any |
kind |
The type of cluster. |
string |
ClusterDefinitionComponentVersion
ClusterIdentity
Name |
Description |
Value |
type |
The type of identity used for the cluster. The type 'SystemAssigned, UserAssigned' includes both an implicitly created identity and a set of user assigned identities. |
'None' 'SystemAssigned' 'SystemAssigned, UserAssigned' 'UserAssigned' |
userAssignedIdentities |
The list of user identities associated with the cluster. The user identity dictionary key references will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}'. |
ClusterIdentityUserAssignedIdentities |
ClusterIdentityUserAssignedIdentities
ComputeIsolationProperties
Name |
Description |
Value |
enableComputeIsolation |
The flag indicates whether enable compute isolation or not. |
bool |
hostSku |
The host sku. |
string |
ComputeProfile
Name |
Description |
Value |
roles |
The list of roles in the cluster. |
Role[] |
DataDisksGroups
Name |
Description |
Value |
disksPerNode |
The number of disks per node. |
int |
DiskEncryptionProperties
Name |
Description |
Value |
encryptionAlgorithm |
Algorithm identifier for encryption, default RSA-OAEP. |
'RSA-OAEP' 'RSA-OAEP-256' 'RSA1_5' |
encryptionAtHost |
Indicates whether or not resource disk encryption is enabled. |
bool |
keyName |
Key name that is used for enabling disk encryption. |
string |
keyVersion |
Specific key version that is used for enabling disk encryption. |
string |
msiResourceId |
Resource ID of Managed Identity that is used to access the key vault. |
string |
vaultUri |
Base key vault URI where the customers key is located eg. https://myvault.vault.azure.net |
string |
EncryptionInTransitProperties
Name |
Description |
Value |
isEncryptionInTransitEnabled |
Indicates whether or not inter cluster node communication is encrypted in transit. |
bool |
HardwareProfile
Name |
Description |
Value |
vmSize |
The size of the VM |
string |
IPConfiguration
Name |
Description |
Value |
name |
The name of private link IP configuration. |
string (required) |
properties |
The private link ip configuration properties. |
IPConfigurationProperties |
IPConfigurationProperties
Name |
Description |
Value |
primary |
Indicates whether this IP configuration is primary for the corresponding NIC. |
bool |
privateIPAddress |
The IP address. |
string |
privateIPAllocationMethod |
The method that private IP address is allocated. |
'dynamic' 'static' |
subnet |
The subnet resource id. |
ResourceId |
IpTag
Name |
Description |
Value |
ipTagType |
Gets or sets the ipTag type: Example FirstPartyUsage. |
string (required) |
tag |
Gets or sets value of the IpTag associated with the public IP. Example HDInsight, SQL, Storage etc |
string (required) |
KafkaRestProperties
KafkaRestPropertiesConfigurationOverride
LinuxOperatingSystemProfile
Name |
Description |
Value |
password |
The password. |
string |
sshProfile |
The SSH profile. |
SshProfile |
username |
The username. |
string |
Microsoft.HDInsight/clusters
Name |
Description |
Value |
apiVersion |
The api version |
'2024-08-01-preview' |
identity |
The identity of the cluster, if configured. |
ClusterIdentity |
location |
The location of the cluster. |
string |
name |
The resource name |
string (required) |
properties |
The cluster create parameters. |
ClusterCreatePropertiesOrClusterGetProperties |
tags |
Resource tags |
Dictionary of tag names and values. See Tags in templates |
type |
The resource type |
'Microsoft.HDInsight/clusters' |
zones |
The availability zones. |
string[] |
NetworkProperties
Name |
Description |
Value |
outboundDependenciesManagedType |
A value to describe how the outbound dependencies of a HDInsight cluster are managed. 'Managed' means that the outbound dependencies are managed by the HDInsight service. 'External' means that the outbound dependencies are managed by a customer specific solution. |
'External' 'Managed' |
privateLink |
Indicates whether or not private link is enabled. |
'Disabled' 'Enabled' |
publicIpTag |
Gets or sets the IP tag for the public IPs created along with the HDInsight Clusters. |
IpTag |
resourceProviderConnection |
The direction for the resource provider connection. |
'Inbound' 'Outbound' |
OsProfile
PrivateLinkConfiguration
Name |
Description |
Value |
name |
The name of private link configuration. |
string (required) |
properties |
The private link configuration properties. |
PrivateLinkConfigurationProperties (required) |
PrivateLinkConfigurationProperties
Name |
Description |
Value |
groupId |
The HDInsight private linkable sub-resource name to apply the private link configuration to. For example, 'headnode', 'gateway', 'edgenode'. |
string (required) |
ipConfigurations |
The IP configurations for the private link service. |
IPConfiguration[] (required) |
ResourceId
Name |
Description |
Value |
id |
The azure resource id. |
string |
Role
Name |
Description |
Value |
autoscale |
The autoscale configurations. |
Autoscale |
dataDisksGroups |
The data disks groups for the role. |
DataDisksGroups[] |
encryptDataDisks |
Indicates whether encrypt the data disks. |
bool |
hardwareProfile |
The hardware profile. |
HardwareProfile |
minInstanceCount |
The minimum instance count of the cluster. |
int |
name |
The name of the role. |
string |
osProfile |
The operating system profile. |
OsProfile |
scriptActions |
The list of script actions on the role. |
ScriptAction[] |
targetInstanceCount |
The instance count of the cluster. |
int |
virtualNetworkProfile |
The virtual network profile. |
VirtualNetworkProfile |
VMGroupName |
The name of the virtual machine group. |
string |
ScriptAction
Name |
Description |
Value |
name |
The name of the script action. |
string (required) |
parameters |
The parameters for the script provided. |
string (required) |
uri |
The URI to the script. |
string (required) |
SecurityProfile
Name |
Description |
Value |
aaddsResourceId |
The resource ID of the user's Azure Active Directory Domain Service. |
string |
clusterUsersGroupDNs |
Optional. The Distinguished Names for cluster user groups |
string[] |
directoryType |
The directory type. |
'ActiveDirectory' |
domain |
The organization's active directory domain. |
string |
domainUsername |
The domain user account that will have admin privileges on the cluster. |
string |
domainUserPassword |
The domain admin password. |
string |
ldapsUrls |
The LDAPS protocol URLs to communicate with the Active Directory. |
string[] |
msiResourceId |
User assigned identity that has permissions to read and create cluster-related artifacts in the user's AADDS. |
string |
organizationalUnitDN |
The organizational unit within the Active Directory to place the cluster and service accounts. |
string |
SshProfile
Name |
Description |
Value |
publicKeys |
The list of SSH public keys. |
SshPublicKey[] |
SshPublicKey
Name |
Description |
Value |
certificateData |
The certificate for SSH. |
string |
StorageAccount
Name |
Description |
Value |
container |
The container in the storage account, only to be specified for WASB storage accounts. |
string |
enableSecureChannel |
Enable secure channel or not, it's an optional field. Default value is false when cluster version < 5.1 and true when cluster version >= 5.1 , |
bool |
fileshare |
The file share name. |
string |
fileSystem |
The filesystem, only to be specified for Azure Data Lake Storage Gen 2. |
string |
isDefault |
Whether or not the storage account is the default storage account. |
bool |
key |
The storage account access key. |
string |
msiResourceId |
The managed identity (MSI) that is allowed to access the storage account, only to be specified for Azure Data Lake Storage Gen 2. |
string |
name |
The name of the storage account. |
string |
resourceId |
The resource ID of storage account, only to be specified for Azure Data Lake Storage Gen 2. |
string |
saskey |
The shared access signature key. |
string |
StorageProfile
Name |
Description |
Value |
storageaccounts |
The list of storage accounts in the cluster. |
StorageAccount[] |
UserAssignedIdentity
Name |
Description |
Value |
tenantId |
The tenant id of user assigned identity. |
string |
VirtualNetworkProfile
Name |
Description |
Value |
id |
The ID of the virtual network. |
string |
subnet |
The name of the subnet. |
string |
Quickstart templates
The following quickstart templates deploy this resource type.
Template |
Description |
Create HDInsight Linux Cluster and run a script action
|
Template creates an HDInsight Linux cluster in a virtual network and then runs a custom script action on every node and sets environment var. |
Creates an HDInsight cluster running ADAM
|
Creates an HDInsight linux cluster running the genomics analysis platform ADAM |
Creates an HDInsight cluster running Apache Spark 1.4.1
|
Creates an HDInsight linux cluster running Apache Spark 1.4.1. |
Deploy a HDInsight cluster and a SQL database
|
This template allows you to create a HDInsight cluster and a SQL Database for testing Sqoop. |
Deploy a HDInsight cluster with an edge node
|
This template allows you to create an HDInsight cluster running Linux with an empty edge node. For more information, see /azure/hdinsight/hdinsight-apps-use-edge-node |
Deploy a Linux-based HBase cluster in HDInsight
|
This template allows you to create a Linux-based HBase cluster in Azure HDInsight. |
Deploy a secure VNet and a HDInsight cluster within the VNet
|
This template allows you to create an Azure VNet and an HDInsight Hadoop cluster running Linux within the VNet. |
Deploy a Spark cluster in a VNet
|
This template allows you to create an Azure VNet and an HDInsight Spark cluster within the VNet. |
Deploy a Spark cluster in Azure HDInsight
|
This template allows you to create a Spark cluster in Azure HDInsight. |
Deploy a VNet, and a HBase cluster within the VNet
|
This template allows you to create an Azure VNet and an HDInsight HBase cluster running Linux within the VNet. |
Deploy an Azure VNet and two HBase clusters within the VNet
|
This template allows you to configure an HBase environment with two HBase clusters within a VNet for configuring HBase replication. |
Deploy an Interative Hive cluster in HDInsight
|
This template allows you to create an Interative Hive (LLAP) cluster in HDInsight and the dependent Azure Storage account. The SSH authentication method for the cluster is username and password. For a template using SSH public key authentication, see /samples/azure/azure-quickstart-templates/hdinsight-linux-ssh-publickey |
Deploy an R-server HDInsight cluster
|
This template allows you to create an HDInsight cluster running Linux with R Server for HDInsight. This template also creates an Azure Storage account. The SSH authentication method for the cluster is username / password. |
Deploy HBase replication with two VNets in one region
|
This template allows you to configure aN HBase environment with two HBase clusters within two VNets in the same region for configuring HBase replication. |
Deploy HDInsight cluster + Confluent Schema Registry node
|
This template allows you to create an HDInsight cluster running Linux with a schema registry edge node. For more information, see /azure/hdinsight/hdinsight-apps-use-edge-node |
Deploy HDInsight cluster with existing default storage
|
This template allows you to create an Hadoop cluster in HDInsight. The cluster uses an existing storage account as the default storage accout. |
Deploy HDInsight cluster with existing linked storage
|
This template allows you to create an Hadoop cluster in HDInsight and the dependent default storage account. The template also links an existing storage account. The linked storage account usually contains the business data. |
Deploy HDInsight cluster with Storage and SSH password
|
This template allows you to create a Linux-based Hadoop cluster in HDInsight and the dependent Azure Storage account. The SSH authentication method for the cluster is username and password. For a template using SSH public key authentication, see /samples/azure/azure-quickstart-templates/hdinsight-linux-ssh-publickey/ |
Deploy HDInsight on Linux (w/ Azure Storage, SSH key)
|
This template allows you to create an HDInsight cluster running Linux. This template also creates an Azure Storage account. The SSH authentication method for the cluster is username / public key. |
Deploy HDInsight on new Data Lake Store and Storage
|
This template allows you to deploy a new Linux HDInsight cluster with new Data Lake Store and Storage accounts. |
Deploy Kafka on HDInsight in a virtual network
|
This template allows you to create an Azure Virtual Network and a Kafka on HDInsight cluster in the virtual network. The SSH authentication method for the cluster is username and password. For a template using SSH public key authentication, see /samples/azure/azure-quickstart-templates/hdinsight-linux-ssh-publickey/ |
Deploy Linux HBase cluster with enhanced writes in HDInsight
|
This template allows you to create a Linux-based HBase cluster with enhanced writes in Azure HDInsight. |
HDInsight (Linux on existing Hive metastore, SSH, vnet)
|
This template allows you to create an HDInsight cluster running Linux, on an existing Hive metastore and virtual network. The SSH authentication method for the cluster is username / password. |
HDInsight cluster with TLS version 1.2 or newer
|
This template allows you to create a HDInsight cluster with TLS protocol enforced to 1.2 or newer. |
HDInsight with custom Ambari + Hive Metastore DB in VNET
|
This template allows you to create an HDInsight cluster in an existing virtual network with a new SQL DB that serves as both a custom Ambari DB and Hive Metastore. You must have an existing SQL Sever, storage account, and VNET. |
HDInsight with Load-based Autoscale Enabled
|
This template allows you to create an HDInsight Spark cluster with load-based Autoscale enabled. |
HDInsight with schedule-based Autoscale Enabled
|
This template allows you to create an HDInsight Spark cluster with schedule-based Autoscale enabled. |
The clusters resource type can be deployed with operations that target:
For a list of changed properties in each API version, see change log.
To create a Microsoft.HDInsight/clusters resource, add the following Terraform to your template.
resource "azapi_resource" "symbolicname" {
type = "Microsoft.HDInsight/clusters@2024-08-01-preview"
name = "string"
identity = {
type = "string"
userAssignedIdentities = {
{customized property} = {
tenantId = "string"
}
}
}
location = "string"
tags = {
{customized property} = "string"
}
zones = [
"string"
]
body = jsonencode({
properties = {
clusterDefinition = {
blueprint = "string"
componentVersion = {
{customized property} = "string"
}
configurations = ?
kind = "string"
}
clusterVersion = "string"
computeIsolationProperties = {
enableComputeIsolation = bool
hostSku = "string"
}
computeProfile = {
roles = [
{
autoscale = {
capacity = {
maxInstanceCount = int
minInstanceCount = int
}
recurrence = {
schedule = [
{
days = [
"string"
]
timeAndCapacity = {
maxInstanceCount = int
minInstanceCount = int
time = "string"
}
}
]
timeZone = "string"
}
}
dataDisksGroups = [
{
disksPerNode = int
}
]
encryptDataDisks = bool
hardwareProfile = {
vmSize = "string"
}
minInstanceCount = int
name = "string"
osProfile = {
linuxOperatingSystemProfile = {
password = "string"
sshProfile = {
publicKeys = [
{
certificateData = "string"
}
]
}
username = "string"
}
}
scriptActions = [
{
name = "string"
parameters = "string"
uri = "string"
}
]
targetInstanceCount = int
virtualNetworkProfile = {
id = "string"
subnet = "string"
}
VMGroupName = "string"
}
]
}
diskEncryptionProperties = {
encryptionAlgorithm = "string"
encryptionAtHost = bool
keyName = "string"
keyVersion = "string"
msiResourceId = "string"
vaultUri = "string"
}
encryptionInTransitProperties = {
isEncryptionInTransitEnabled = bool
}
kafkaRestProperties = {
clientGroupInfo = {
groupId = "string"
groupName = "string"
}
configurationOverride = {
{customized property} = "string"
}
}
minSupportedTlsVersion = "string"
networkProperties = {
outboundDependenciesManagedType = "string"
privateLink = "string"
publicIpTag = {
ipTagType = "string"
tag = "string"
}
resourceProviderConnection = "string"
}
osType = "string"
privateLinkConfigurations = [
{
name = "string"
properties = {
groupId = "string"
ipConfigurations = [
{
name = "string"
properties = {
primary = bool
privateIPAddress = "string"
privateIPAllocationMethod = "string"
subnet = {
id = "string"
}
}
}
]
}
}
]
securityProfile = {
aaddsResourceId = "string"
clusterUsersGroupDNs = [
"string"
]
directoryType = "string"
domain = "string"
domainUsername = "string"
domainUserPassword = "string"
ldapsUrls = [
"string"
]
msiResourceId = "string"
organizationalUnitDN = "string"
}
storageProfile = {
storageaccounts = [
{
container = "string"
enableSecureChannel = bool
fileshare = "string"
fileSystem = "string"
isDefault = bool
key = "string"
msiResourceId = "string"
name = "string"
resourceId = "string"
saskey = "string"
}
]
}
tier = "string"
}
})
}
Property values
Autoscale
AutoscaleCapacity
Name |
Description |
Value |
maxInstanceCount |
The maximum instance count of the cluster |
int |
minInstanceCount |
The minimum instance count of the cluster |
int |
AutoscaleRecurrence
Name |
Description |
Value |
schedule |
Array of schedule-based autoscale rules |
AutoscaleSchedule[] |
timeZone |
The time zone for the autoscale schedule times |
string |
AutoscaleSchedule
Name |
Description |
Value |
days |
Days of the week for a schedule-based autoscale rule |
String array containing any of: 'Friday' 'Monday' 'Saturday' 'Sunday' 'Thursday' 'Tuesday' 'Wednesday' |
timeAndCapacity |
Time and capacity for a schedule-based autoscale rule |
AutoscaleTimeAndCapacity |
AutoscaleTimeAndCapacity
Name |
Description |
Value |
maxInstanceCount |
The maximum instance count of the cluster |
int |
minInstanceCount |
The minimum instance count of the cluster |
int |
time |
24-hour time in the form xx:xx |
string |
ClientGroupInfo
Name |
Description |
Value |
groupId |
The AAD security group id. |
string |
groupName |
The AAD security group name. |
string |
ClusterCreatePropertiesOrClusterGetProperties
ClusterDefinition
Name |
Description |
Value |
blueprint |
The link to the blueprint. |
string |
componentVersion |
The versions of different services in the cluster. |
ClusterDefinitionComponentVersion |
configurations |
The cluster configurations. |
any |
kind |
The type of cluster. |
string |
ClusterDefinitionComponentVersion
ClusterIdentity
Name |
Description |
Value |
type |
The type of identity used for the cluster. The type 'SystemAssigned, UserAssigned' includes both an implicitly created identity and a set of user assigned identities. |
'None' 'SystemAssigned' 'SystemAssigned, UserAssigned' 'UserAssigned' |
userAssignedIdentities |
The list of user identities associated with the cluster. The user identity dictionary key references will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}'. |
ClusterIdentityUserAssignedIdentities |
ClusterIdentityUserAssignedIdentities
ComputeIsolationProperties
Name |
Description |
Value |
enableComputeIsolation |
The flag indicates whether enable compute isolation or not. |
bool |
hostSku |
The host sku. |
string |
ComputeProfile
Name |
Description |
Value |
roles |
The list of roles in the cluster. |
Role[] |
DataDisksGroups
Name |
Description |
Value |
disksPerNode |
The number of disks per node. |
int |
DiskEncryptionProperties
Name |
Description |
Value |
encryptionAlgorithm |
Algorithm identifier for encryption, default RSA-OAEP. |
'RSA-OAEP' 'RSA-OAEP-256' 'RSA1_5' |
encryptionAtHost |
Indicates whether or not resource disk encryption is enabled. |
bool |
keyName |
Key name that is used for enabling disk encryption. |
string |
keyVersion |
Specific key version that is used for enabling disk encryption. |
string |
msiResourceId |
Resource ID of Managed Identity that is used to access the key vault. |
string |
vaultUri |
Base key vault URI where the customers key is located eg. https://myvault.vault.azure.net |
string |
EncryptionInTransitProperties
Name |
Description |
Value |
isEncryptionInTransitEnabled |
Indicates whether or not inter cluster node communication is encrypted in transit. |
bool |
HardwareProfile
Name |
Description |
Value |
vmSize |
The size of the VM |
string |
IPConfiguration
Name |
Description |
Value |
name |
The name of private link IP configuration. |
string (required) |
properties |
The private link ip configuration properties. |
IPConfigurationProperties |
IPConfigurationProperties
Name |
Description |
Value |
primary |
Indicates whether this IP configuration is primary for the corresponding NIC. |
bool |
privateIPAddress |
The IP address. |
string |
privateIPAllocationMethod |
The method that private IP address is allocated. |
'dynamic' 'static' |
subnet |
The subnet resource id. |
ResourceId |
IpTag
Name |
Description |
Value |
ipTagType |
Gets or sets the ipTag type: Example FirstPartyUsage. |
string (required) |
tag |
Gets or sets value of the IpTag associated with the public IP. Example HDInsight, SQL, Storage etc |
string (required) |
KafkaRestProperties
KafkaRestPropertiesConfigurationOverride
LinuxOperatingSystemProfile
Name |
Description |
Value |
password |
The password. |
string |
sshProfile |
The SSH profile. |
SshProfile |
username |
The username. |
string |
Microsoft.HDInsight/clusters
Name |
Description |
Value |
identity |
The identity of the cluster, if configured. |
ClusterIdentity |
location |
The location of the cluster. |
string |
name |
The resource name |
string (required) |
properties |
The cluster create parameters. |
ClusterCreatePropertiesOrClusterGetProperties |
tags |
Resource tags |
Dictionary of tag names and values. |
type |
The resource type |
"Microsoft.HDInsight/clusters@2024-08-01-preview" |
zones |
The availability zones. |
string[] |
NetworkProperties
Name |
Description |
Value |
outboundDependenciesManagedType |
A value to describe how the outbound dependencies of a HDInsight cluster are managed. 'Managed' means that the outbound dependencies are managed by the HDInsight service. 'External' means that the outbound dependencies are managed by a customer specific solution. |
'External' 'Managed' |
privateLink |
Indicates whether or not private link is enabled. |
'Disabled' 'Enabled' |
publicIpTag |
Gets or sets the IP tag for the public IPs created along with the HDInsight Clusters. |
IpTag |
resourceProviderConnection |
The direction for the resource provider connection. |
'Inbound' 'Outbound' |
OsProfile
PrivateLinkConfiguration
Name |
Description |
Value |
name |
The name of private link configuration. |
string (required) |
properties |
The private link configuration properties. |
PrivateLinkConfigurationProperties (required) |
PrivateLinkConfigurationProperties
Name |
Description |
Value |
groupId |
The HDInsight private linkable sub-resource name to apply the private link configuration to. For example, 'headnode', 'gateway', 'edgenode'. |
string (required) |
ipConfigurations |
The IP configurations for the private link service. |
IPConfiguration[] (required) |
ResourceId
Name |
Description |
Value |
id |
The azure resource id. |
string |
Role
Name |
Description |
Value |
autoscale |
The autoscale configurations. |
Autoscale |
dataDisksGroups |
The data disks groups for the role. |
DataDisksGroups[] |
encryptDataDisks |
Indicates whether encrypt the data disks. |
bool |
hardwareProfile |
The hardware profile. |
HardwareProfile |
minInstanceCount |
The minimum instance count of the cluster. |
int |
name |
The name of the role. |
string |
osProfile |
The operating system profile. |
OsProfile |
scriptActions |
The list of script actions on the role. |
ScriptAction[] |
targetInstanceCount |
The instance count of the cluster. |
int |
virtualNetworkProfile |
The virtual network profile. |
VirtualNetworkProfile |
VMGroupName |
The name of the virtual machine group. |
string |
ScriptAction
Name |
Description |
Value |
name |
The name of the script action. |
string (required) |
parameters |
The parameters for the script provided. |
string (required) |
uri |
The URI to the script. |
string (required) |
SecurityProfile
Name |
Description |
Value |
aaddsResourceId |
The resource ID of the user's Azure Active Directory Domain Service. |
string |
clusterUsersGroupDNs |
Optional. The Distinguished Names for cluster user groups |
string[] |
directoryType |
The directory type. |
'ActiveDirectory' |
domain |
The organization's active directory domain. |
string |
domainUsername |
The domain user account that will have admin privileges on the cluster. |
string |
domainUserPassword |
The domain admin password. |
string |
ldapsUrls |
The LDAPS protocol URLs to communicate with the Active Directory. |
string[] |
msiResourceId |
User assigned identity that has permissions to read and create cluster-related artifacts in the user's AADDS. |
string |
organizationalUnitDN |
The organizational unit within the Active Directory to place the cluster and service accounts. |
string |
SshProfile
Name |
Description |
Value |
publicKeys |
The list of SSH public keys. |
SshPublicKey[] |
SshPublicKey
Name |
Description |
Value |
certificateData |
The certificate for SSH. |
string |
StorageAccount
Name |
Description |
Value |
container |
The container in the storage account, only to be specified for WASB storage accounts. |
string |
enableSecureChannel |
Enable secure channel or not, it's an optional field. Default value is false when cluster version < 5.1 and true when cluster version >= 5.1 , |
bool |
fileshare |
The file share name. |
string |
fileSystem |
The filesystem, only to be specified for Azure Data Lake Storage Gen 2. |
string |
isDefault |
Whether or not the storage account is the default storage account. |
bool |
key |
The storage account access key. |
string |
msiResourceId |
The managed identity (MSI) that is allowed to access the storage account, only to be specified for Azure Data Lake Storage Gen 2. |
string |
name |
The name of the storage account. |
string |
resourceId |
The resource ID of storage account, only to be specified for Azure Data Lake Storage Gen 2. |
string |
saskey |
The shared access signature key. |
string |
StorageProfile
Name |
Description |
Value |
storageaccounts |
The list of storage accounts in the cluster. |
StorageAccount[] |
UserAssignedIdentity
Name |
Description |
Value |
tenantId |
The tenant id of user assigned identity. |
string |
VirtualNetworkProfile
Name |
Description |
Value |
id |
The ID of the virtual network. |
string |
subnet |
The name of the subnet. |
string |