Share via

Clusters - Create

  • Reference
Service:
HDInsight
API Version:
2021-06-01

Creates a new HDInsight cluster with the specified parameters.

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.HDInsight/clusters/{clusterName}?api-version=2021-06-01

URI Parameters

Name In Required Type Description
clusterName
 path True

string

The name of the cluster.
resourceGroupName
 path True

string

The name of the resource group.
subscriptionId
 path True

string

The subscription credentials which uniquely identify Microsoft Azure subscription. The subscription ID forms part of the URI for every service call.
api-version
 query True

string

The HDInsight client API Version.

Request Body

Name Type Description
identity

ClusterIdentity

The identity of the cluster, if configured.
location

string

The location of the cluster.
properties

ClusterCreateProperties

The cluster create parameters.
tags

object

The resource tags.
zones

string[]

The availability zones.

Responses

Name Type Description
200 OK

Cluster

OK response definition.
Other Status Codes

ErrorResponse

Error response describing why the operation failed.

Security

azure_auth

Azure Active Directory OAuth2 Flow

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize

Scopes

Name Description
user_impersonation impersonate your user account

Examples

Create cluster with availability zones
Create cluster with compute isolation properties
Create cluster with encryption at host
Create cluster with encryption in transit
Create cluster with network properties
Create cluster with TLS 1.2
Create Hadoop cluster with Azure Data Lake Storage Gen 2
Create Hadoop on Linux cluster with SSH password
Create Hadoop on Linux cluster with SSH public key
Create HDInsight cluster with Autoscale configuration
Create Kafka cluster with Kafka Rest Proxy
Create Secure Hadoop cluster
Create Spark on Linux Cluster with SSH password

Create cluster with availability zones

Sample request

PUT https://management.azure.com/subscriptions/subId/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "zones": [
    "1"
  ],
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "clusterDefinition": {
      "kind": "hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        },
        "ambari-conf": {
          "database-server": "{sql server name}.database.windows.net",
          "database-name": "{ambari database name}",
          "database-user-name": "**********",
          "database-user-password": "**********"
        },
        "hive-site": {
          "javax.jdo.option.ConnectionURL": "jdbc:sqlserver://{sql server name}.database.windows.net;database={hive metastore name};encrypt=true;trustServerCertificate=true;create=false;loginTimeout=300;sendStringParametersAsUnicode=true;prepareSQL=0",
          "javax.jdo.option.ConnectionUserName": "**********",
          "javax.jdo.option.ConnectionPassword": "**********!",
          "javax.jdo.option.ConnectionDriverName": "com.microsoft.sqlserver.jdbc.SQLServerDriver"
        },
        "hive-env": {
          "hive_database": "Existing MSSQL Server database with SQL authentication",
          "hive_database_name": "{hive metastore name}",
          "hive_database_type": "mssql",
          "hive_existing_mssql_server_database": "{hive metastore name}",
          "hive_existing_mssql_server_host": "{sql server name}.database.windows.net",
          "hive_hostname": "{sql server name}.database.windows.net"
        },
        "oozie-site": {
          "oozie.service.JPAService.jdbc.url": "jdbc:sqlserver://{sql server name}.database.windows.net;database={oozie metastore name};encrypt=true;trustServerCertificate=true;create=false;loginTimeout=300;sendStringParametersAsUnicode=true;prepareSQL=0",
          "oozie.service.JPAService.jdbc.username": "**********",
          "oozie.service.JPAService.jdbc.password": "**********",
          "oozie.service.JPAService.jdbc.driver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
          "oozie.db.schema.name": "oozie"
        },
        "oozie-env": {
          "oozie_database": "Existing MSSQL Server database with SQL authentication",
          "oozie_database_name": "{oozie metastore name}",
          "oozie_database_type": "mssql",
          "oozie_existing_mssql_server_database": "{oozie metastore name}",
          "oozie_existing_mssql_server_host": "{sql server name}.database.windows.net",
          "oozie_hostname": "{sql server name}.database.windows.net"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage",
          "isDefault": true,
          "container": "containername",
          "key": "storage account key"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "South Central US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "tags": null,
  "zones": [
    "1"
  ],
  "properties": {
    "clusterVersion": "3.6.1000.67",
    "osType": "Linux",
    "clusterId": "8186508b6234470e9d16c9e8e13bd821",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-3.6.1000.67.2005040905.json",
      "kind": "hadoop"
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2020-06-09T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 16
    },
    "tier": "Standard"
  }
}

Create cluster with compute isolation properties

Sample request

PUT https://management.azure.com/subscriptions/subId/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "clusterDefinition": {
      "kind": "hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage",
          "isDefault": true,
          "container": "containername",
          "key": "storage account key"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          }
        }
      ]
    },
    "computeIsolationProperties": {
      "enableComputeIsolation": true,
      "hostSku": null
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "South Central US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "tags": null,
  "properties": {
    "clusterVersion": "3.6.1000.67",
    "osType": "Linux",
    "clusterId": "8186508b6234470e9d16c9e8e13bd821",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-3.6.1000.67.2005040905.json",
      "kind": "hadoop"
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "computeIsolationProperties": {
      "enableComputeIsolation": true,
      "hostSku": null
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2020-06-09T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 16
    },
    "tier": "Standard"
  }
}

Create cluster with encryption at host

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "Hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_DS14_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Standard_DS14_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Standard_DS14_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        }
      ]
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "default8525",
          "key": "storagekey"
        }
      ]
    },
    "diskEncryptionProperties": {
      "encryptionAtHost": true
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "South Central US",
  "etag": "3b76ce3d-892c-4036-9d8b-8ade18ba7a4b",
  "tags": null,
  "properties": {
    "clusterVersion": "3.6.1000.67",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-3.6.1000.67.2001080246.json",
      "kind": "Hadoop",
      "componentVersion": {
        "Hadoop": "2.7"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_ds14_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_ds14_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_ds14_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2020-01-10T08:36:39.153",
    "quotaInfo": {
      "coresUsed": 20
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard",
    "diskEncryptionProperties": {
      "vaultUri": null,
      "keyName": null,
      "keyVersion": null,
      "encryptionAlgorithm": null,
      "msiResourceId": null,
      "encryptionAtHost": true
    }
  }
}

Create cluster with encryption in transit

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "Hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Large"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Large"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Small"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        }
      ]
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "default8525",
          "key": "storagekey"
        }
      ]
    },
    "encryptionInTransitProperties": {
      "isEncryptionInTransitEnabled": true
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "South Central US",
  "etag": "3b76ce3d-892c-4036-9d8b-8ade18ba7a4b",
  "tags": null,
  "properties": {
    "clusterVersion": "3.6.1000.67",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-3.6.1000.67.2001080246.json",
      "kind": "Hadoop",
      "componentVersion": {
        "Hadoop": "2.7"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_a4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2020-01-10T08:36:39.153",
    "quotaInfo": {
      "coresUsed": 20
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard",
    "encryptionInTransitProperties": {
      "isEncryptionInTransitEnabled": true
    }
  }
}

Create cluster with network properties

Sample request

PUT https://management.azure.com/subscriptions/subId/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "clusterDefinition": {
      "kind": "hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage",
          "isDefault": true,
          "container": "containername",
          "key": "storage account key"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        }
      ]
    },
    "networkProperties": {
      "resourceProviderConnection": "Outbound",
      "privateLink": "Enabled"
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "South Central US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "tags": null,
  "properties": {
    "clusterVersion": "3.6.1000.67",
    "osType": "Linux",
    "clusterId": "8186508b6234470e9d16c9e8e13bd821",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-3.6.1000.67.2005040905.json",
      "kind": "hadoop"
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          }
        }
      ]
    },
    "networkProperties": {
      "resourceProviderConnection": "Outbound",
      "privateLink": "Enabled"
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2020-06-09T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 16
    },
    "tier": "Standard"
  }
}

Create cluster with TLS 1.2

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "Hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Large"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Large"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Small"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        }
      ]
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "default8525",
          "key": "storagekey"
        }
      ]
    },
    "minSupportedTlsVersion": "1.2"
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "South Central US",
  "etag": "3b76ce3d-892c-4036-9d8b-8ade18ba7a4b",
  "tags": null,
  "properties": {
    "clusterVersion": "3.6.1000.67",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-3.6.1000.67.2001080246.json",
      "kind": "Hadoop",
      "componentVersion": {
        "Hadoop": "2.7"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_a4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2020-01-10T08:36:39.153",
    "quotaInfo": {
      "coresUsed": 20
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard",
    "minSupportedTlsVersion": "1.2"
  }
}

Create Hadoop cluster with Azure Data Lake Storage Gen 2

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "tags": {
    "key1": "val1"
  },
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "Hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": "true",
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.dfs.core.windows.net",
          "isDefault": true,
          "fileSystem": "default",
          "key": "storagekey"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "minInstanceCount": 1,
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "workernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "zookeepernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Small"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "East US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "properties": {
    "clusterVersion": "4.0.1000.1",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-4.0.1000.1.1910270459.json",
      "kind": "HADOOP",
      "componentVersion": {
        "Hadoop": "3.1"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d12_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "standard_d4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2019-11-18T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 40
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard"
  }
}

Create Hadoop on Linux cluster with SSH password

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "tags": {
    "key1": "val1"
  },
  "properties": {
    "clusterVersion": "3.5",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "Hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": "true",
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "containername",
          "key": "storagekey"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "minInstanceCount": 1,
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "workernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "zookeepernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Small"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "East US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "properties": {
    "clusterVersion": "4.0.1000.1",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-4.0.1000.1.1910270459.json",
      "kind": "HADOOP",
      "componentVersion": {
        "Hadoop": "3.1"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d12_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "standard_d4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2019-11-18T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 40
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard"
  }
}

Create Hadoop on Linux cluster with SSH public key

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "tags": {
    "key1": "val1"
  },
  "properties": {
    "clusterVersion": "3.5",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "Hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "containername",
          "key": "storagekey"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "minInstanceCount": 1,
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          }
        },
        {
          "name": "workernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "zookeepernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Small"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "East US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "properties": {
    "clusterVersion": "4.0.1000.1",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-4.0.1000.1.1910270459.json",
      "kind": "HADOOP",
      "componentVersion": {
        "Hadoop": "3.1"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d12_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "standard_d4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2019-11-18T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 40
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard"
  }
}

Create HDInsight cluster with Autoscale configuration

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "properties": {
    "clusterVersion": "3.6",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "hadoop",
      "componentVersion": {
        "Hadoop": "2.7"
      },
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "hdinsight-autoscale-tes-2019-06-18t05-49-16-591z",
          "key": "storagekey"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "autoscale": {
            "capacity": null,
            "recurrence": {
              "timeZone": "China Standard Time",
              "schedule": [
                {
                  "days": [
                    "Monday",
                    "Tuesday",
                    "Wednesday",
                    "Thursday",
                    "Friday"
                  ],
                  "timeAndCapacity": {
                    "time": "09:00",
                    "minInstanceCount": 3,
                    "maxInstanceCount": 3
                  }
                },
                {
                  "days": [
                    "Monday",
                    "Tuesday",
                    "Wednesday",
                    "Thursday",
                    "Friday"
                  ],
                  "timeAndCapacity": {
                    "time": "18:00",
                    "minInstanceCount": 6,
                    "maxInstanceCount": 6
                  }
                },
                {
                  "days": [
                    "Saturday",
                    "Sunday"
                  ],
                  "timeAndCapacity": {
                    "time": "09:00",
                    "minInstanceCount": 2,
                    "maxInstanceCount": 2
                  }
                },
                {
                  "days": [
                    "Saturday",
                    "Sunday"
                  ],
                  "timeAndCapacity": {
                    "time": "18:00",
                    "minInstanceCount": 4,
                    "maxInstanceCount": 4
                  }
                }
              ]
            }
          },
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "Standard_D4_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          },
          "virtualNetworkProfile": null,
          "scriptActions": [],
          "dataDisksGroups": null
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "East US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "properties": {
    "clusterVersion": "4.0.1000.1",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-4.0.1000.1.1910270459.json",
      "kind": "HADOOP",
      "componentVersion": {
        "Hadoop": "3.1"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d12_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "standard_d4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2019-11-18T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 40
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard"
  }
}

Create Kafka cluster with Kafka Rest Proxy

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "properties": {
    "clusterVersion": "4.0",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "kafka",
      "componentVersion": {
        "Kafka": "2.1"
      },
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "kafkaRestProperties": {
      "clientGroupInfo": {
        "groupName": "Kafka security group name",
        "groupId": "00000000-0000-0000-0000-111111111111"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Large"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Large"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          },
          "dataDisksGroups": [
            {
              "disksPerNode": 8
            }
          ]
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Small"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "kafkamanagementnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_D4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "kafkauser",
              "password": "**********"
            }
          }
        }
      ]
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "containername",
          "key": "storagekey"
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "South Central US",
  "etag": "e1266b83-9bda-4797-a906-1bf82c8eb09a",
  "tags": null,
  "properties": {
    "clusterVersion": "4.0.1000.1",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/kafka-4.0.1000.1.1911212244.json",
      "kind": "KAFKA",
      "componentVersion": {
        "Kafka": "2.1"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d3_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "standard_d3_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          },
          "dataDisksGroups": [
            {
              "disksPerNode": 2,
              "storageAccountType": "Standard_LRS",
              "diskSizeGB": 1023
            }
          ]
        },
        {
          "name": "kafkamanagementnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2019-11-25T03:43:23.663",
    "quotaInfo": {
      "coresUsed": 52
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      },
      {
        "name": "KafkaRestProxyPublicEndpoint",
        "protocol": "TCP",
        "location": "cluster1-kafkarest.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard",
    "kafkaRestProperties": {
      "clientGroupInfo": {
        "groupName": "security group name",
        "groupId": "00000000-0000-0000-0000-000000000000"
      }
    }
  }
}

Create Secure Hadoop cluster

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "tags": {
    "key1": "val1"
  },
  "properties": {
    "clusterVersion": "3.5",
    "osType": "Linux",
    "tier": "Premium",
    "clusterDefinition": {
      "kind": "Hadoop",
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "containername",
          "key": "storage account key"
        }
      ]
    },
    "securityProfile": {
      "directoryType": "ActiveDirectory",
      "domain": "DomainName",
      "organizationalUnitDN": "OU=Hadoop,DC=hdinsight,DC=test",
      "ldapsUrls": [
        "ldaps://10.10.0.4:636"
      ],
      "domainUsername": "DomainUsername",
      "domainUserPassword": "**********",
      "clusterUsersGroupDNs": [
        "hdiusers"
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "minInstanceCount": 1,
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          },
          "scriptActions": []
        },
        {
          "name": "workernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "Standard_D3_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          },
          "scriptActions": []
        },
        {
          "name": "zookeepernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "Small"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********",
              "sshProfile": {
                "publicKeys": [
                  {
                    "certificateData": "**********"
                  }
                ]
              }
            }
          },
          "virtualNetworkProfile": {
            "id": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname",
            "subnet": "/subscriptions/subId/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnetname/subnets/vnetsubnet"
          },
          "scriptActions": []
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "East US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "properties": {
    "clusterVersion": "4.0.1000.1",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-4.0.1000.1.1910270459.json",
      "kind": "HADOOP",
      "componentVersion": {
        "Hadoop": "3.1"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d12_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "standard_d4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2019-11-18T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 40
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard"
  }
}

Create Spark on Linux Cluster with SSH password

Sample request

PUT https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1?api-version=2021-06-01

{
  "tags": {
    "key1": "val1"
  },
  "properties": {
    "clusterVersion": "3.5",
    "osType": "Linux",
    "tier": "Standard",
    "clusterDefinition": {
      "kind": "Spark",
      "componentVersion": {
        "Spark": "2.0"
      },
      "configurations": {
        "gateway": {
          "restAuthCredential.isEnabled": true,
          "restAuthCredential.username": "admin",
          "restAuthCredential.password": "**********"
        }
      }
    },
    "storageProfile": {
      "storageaccounts": [
        {
          "name": "mystorage.blob.core.windows.net",
          "isDefault": true,
          "container": "containername",
          "key": "storageapikey*"
        }
      ]
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "minInstanceCount": 1,
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "Standard_D12_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        },
        {
          "name": "workernode",
          "minInstanceCount": 1,
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "Standard_D4_V2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser",
              "password": "**********"
            }
          }
        }
      ]
    }
  }
}

Sample response

Status code:
200 
location: https://management.azure.com/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1/azureasyncoperations/create?api-version=2021-06-01
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.HDInsight/clusters/cluster1",
  "name": "cluster1",
  "type": "Microsoft.HDInsight/clusters",
  "location": "East US",
  "etag": "fdf2a6e8-ce83-42cc-8c2d-0ceb11a370ff",
  "properties": {
    "clusterVersion": "4.0.1000.1",
    "osType": "Linux",
    "clusterDefinition": {
      "blueprint": "https://blueprints.azurehdinsight.net/hadoop-4.0.1000.1.1910270459.json",
      "kind": "SPARK",
      "componentVersion": {
        "Hadoop": "3.1"
      }
    },
    "computeProfile": {
      "roles": [
        {
          "name": "headnode",
          "targetInstanceCount": 2,
          "hardwareProfile": {
            "vmSize": "standard_d12_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "workernode",
          "targetInstanceCount": 4,
          "hardwareProfile": {
            "vmSize": "standard_d4_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        },
        {
          "name": "zookeepernode",
          "targetInstanceCount": 3,
          "hardwareProfile": {
            "vmSize": "standard_a2_v2"
          },
          "osProfile": {
            "linuxOperatingSystemProfile": {
              "username": "sshuser"
            }
          }
        }
      ]
    },
    "provisioningState": "Succeeded",
    "clusterState": "Running",
    "createdDate": "2019-11-18T12:25:43.48",
    "quotaInfo": {
      "coresUsed": 40
    },
    "connectivityEndpoints": [
      {
        "name": "SSH",
        "protocol": "TCP",
        "location": "cluster1-ssh.azurehdinsight.net",
        "port": 22
      },
      {
        "name": "HTTPS",
        "protocol": "TCP",
        "location": "cluster1.azurehdinsight.net",
        "port": 443
      }
    ],
    "tier": "Standard"
  }
}

Definitions

Name Description
Autoscale

The autoscale request parameters
AutoscaleCapacity

The load-based autoscale request parameters
AutoscaleRecurrence

Schedule-based autoscale request parameters
AutoscaleSchedule

Parameters for a schedule-based autoscale rule, consisting of an array of days + a time and capacity
AutoscaleTimeAndCapacity

Time and capacity request parameters
ClientGroupInfo

The information of AAD security group.
Cluster

The HDInsight cluster.
ClusterCreateParametersExtended

The CreateCluster request parameters.
ClusterCreateProperties

The cluster create parameters.
ClusterDefinition

The cluster definition.
ClusterGetProperties

The properties of cluster.
ClusterIdentity

Identity for the cluster.
ComputeIsolationProperties

The compute isolation properties.
ComputeProfile

Describes the compute profile.
ConnectivityEndpoint

The connectivity properties
createdByType

The type of identity that created the resource.
DataDisksGroups

The data disks groups for the role.
DaysOfWeek

Days of the week for a schedule-based autoscale rule
DirectoryType

The directory type.
DiskEncryptionProperties

The disk encryption properties
EncryptionInTransitProperties

The encryption-in-transit properties.
ErrorResponse

Describes the format of Error response.
Errors

The error message associated with the cluster creation.
ExcludedServicesConfig

The configuration that services will be excluded when creating cluster.
HardwareProfile

The hardware profile.
HDInsightClusterProvisioningState

The provisioning state, which only appears in the response.
IPConfiguration

The ip configurations for the private link service.
JsonWebKeyEncryptionAlgorithm

Algorithm identifier for encryption, default RSA-OAEP.
KafkaRestProperties

The kafka rest proxy configuration which contains AAD security group information.
LinuxOperatingSystemProfile

The ssh username, password, and ssh public key.
NetworkProperties

The network properties.
OsProfile

The Linux operation systems profile.
OSType

The type of operating system.
PrivateEndpoint

The private endpoint.
PrivateEndpointConnection

The private endpoint connection.
PrivateEndpointConnectionProvisioningState

The provisioning state, which only appears in the response.
PrivateIPAllocationMethod

The method that private IP address is allocated.
PrivateLink

Indicates whether or not private link is enabled.
PrivateLinkConfiguration

The private link configuration.
PrivateLinkConfigurationProvisioningState

The private link configuration provisioning state, which only appears in the response.
PrivateLinkServiceConnectionState

The private link service connection state.
PrivateLinkServiceConnectionStatus

The concrete private link service connection.
QuotaInfo

The quota properties for the cluster.
ResourceId

The azure resource id.
ResourceIdentityType

The type of identity used for the cluster. The type 'SystemAssigned, UserAssigned' includes both an implicitly created identity and a set of user assigned identities.
ResourceProviderConnection

The direction for the resource provider connection.
Role

Describes a role on the cluster.
ScriptAction

Describes a script action on role on the cluster.
SecurityProfile

The security profile which contains Ssh public key for the HDInsight cluster.
SshProfile

The list of SSH public keys.
SshPublicKey

The SSH public key for the cluster nodes.
StorageAccount

The storage Account.
StorageProfile

The storage profile.
systemData

Metadata pertaining to creation and last modification of the resource.
Tier

The cluster tier.
UserAssignedIdentity

The User Assigned Identity
VirtualNetworkProfile

The virtual network properties.

Autoscale

The autoscale request parameters

Name Type Description
capacity

AutoscaleCapacity

Parameters for load-based autoscale
recurrence

AutoscaleRecurrence

Parameters for schedule-based autoscale

AutoscaleCapacity

The load-based autoscale request parameters

Name Type Description
maxInstanceCount

integer

The maximum instance count of the cluster
minInstanceCount

integer

The minimum instance count of the cluster

AutoscaleRecurrence

Schedule-based autoscale request parameters

Name Type Description
schedule

AutoscaleSchedule[]

Array of schedule-based autoscale rules
timeZone

string

The time zone for the autoscale schedule times

AutoscaleSchedule

Parameters for a schedule-based autoscale rule, consisting of an array of days + a time and capacity

Name Type Description
days

DaysOfWeek[]

Days of the week for a schedule-based autoscale rule
timeAndCapacity

AutoscaleTimeAndCapacity

Time and capacity for a schedule-based autoscale rule

AutoscaleTimeAndCapacity

Time and capacity request parameters

Name Type Description
maxInstanceCount

integer

The maximum instance count of the cluster
minInstanceCount

integer

The minimum instance count of the cluster
time

string

24-hour time in the form xx:xx

ClientGroupInfo

The information of AAD security group.

Name Type Description
groupId

string

The AAD security group id.
groupName

string

The AAD security group name.

Cluster

The HDInsight cluster.

Name Type Description
etag

string

The ETag for the resource
id

string

Fully qualified resource ID for the resource. Ex - /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}
identity

ClusterIdentity

The identity of the cluster, if configured.
location

string

The geo-location where the resource lives
name

string

The name of the resource
properties

ClusterGetProperties

The properties of the cluster.
systemData

systemData

Metadata pertaining to creation and last modification of the resource.
tags

object

Resource tags.
type

string

The type of the resource. E.g. "Microsoft.Compute/virtualMachines" or "Microsoft.Storage/storageAccounts"
zones

string[]

The availability zones.

ClusterCreateParametersExtended

The CreateCluster request parameters.

Name Type Description
identity

ClusterIdentity

The identity of the cluster, if configured.
location

string

The location of the cluster.
properties

ClusterCreateProperties

The cluster create parameters.
tags

object

The resource tags.
zones

string[]

The availability zones.

ClusterCreateProperties

The cluster create parameters.

Name Type Default value Description
clusterDefinition

ClusterDefinition

The cluster definition.
clusterVersion

string

The version of the cluster.
computeIsolationProperties

ComputeIsolationProperties

The compute isolation properties.
computeProfile

ComputeProfile

The compute profile.
diskEncryptionProperties

DiskEncryptionProperties

The disk encryption properties.
encryptionInTransitProperties

EncryptionInTransitProperties

The encryption-in-transit properties.
kafkaRestProperties

KafkaRestProperties

The cluster kafka rest proxy configuration.
minSupportedTlsVersion

string

The minimal supported tls version.
networkProperties

NetworkProperties

The network properties.
osType

OSType

The type of operating system.
privateLinkConfigurations

PrivateLinkConfiguration[]

The private link configurations.
securityProfile

SecurityProfile

The security profile.
storageProfile

StorageProfile

The storage profile.
tier

Tier

Standard

The cluster tier.

ClusterDefinition

The cluster definition.

Name Type Description
blueprint

string

The link to the blueprint.
componentVersion

object

The versions of different services in the cluster.
configurations

object

The cluster configurations.
kind

string

The type of cluster.

ClusterGetProperties

The properties of cluster.

Name Type Description
clusterDefinition

ClusterDefinition

The cluster definition.
clusterHdpVersion

string

The hdp version of the cluster.
clusterId

string

The cluster id.
clusterState

string

The state of the cluster.
clusterVersion

string

The version of the cluster.
computeIsolationProperties

ComputeIsolationProperties

The compute isolation properties.
computeProfile

ComputeProfile

The compute profile.
connectivityEndpoints

ConnectivityEndpoint[]

The list of connectivity endpoints.
createdDate

string

The date on which the cluster was created.
diskEncryptionProperties

DiskEncryptionProperties

The disk encryption properties.
encryptionInTransitProperties

EncryptionInTransitProperties

The encryption-in-transit properties.
errors

Errors[]

The list of errors.
excludedServicesConfig

ExcludedServicesConfig

The excluded services config.
kafkaRestProperties

KafkaRestProperties

The cluster kafka rest proxy configuration.
minSupportedTlsVersion

string

The minimal supported tls version.
networkProperties

NetworkProperties

The network properties.
osType

OSType

The type of operating system.
privateEndpointConnections

PrivateEndpointConnection[]

The list of private endpoint connections.
privateLinkConfigurations

PrivateLinkConfiguration[]

The private link configurations.
provisioningState

HDInsightClusterProvisioningState

The provisioning state, which only appears in the response.
quotaInfo

QuotaInfo

The quota information.
securityProfile

SecurityProfile

The security profile.
storageProfile

StorageProfile

The storage profile.
tier

Tier

The cluster tier.

ClusterIdentity

Identity for the cluster.

Name Type Description
principalId

string

The principal id of cluster identity. This property will only be provided for a system assigned identity.
tenantId

string

The tenant id associated with the cluster. This property will only be provided for a system assigned identity.
type

ResourceIdentityType

The type of identity used for the cluster. The type 'SystemAssigned, UserAssigned' includes both an implicitly created identity and a set of user assigned identities.
userAssignedIdentities

<string,  UserAssignedIdentity>

The list of user identities associated with the cluster. The user identity dictionary key references will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}'.

ComputeIsolationProperties

The compute isolation properties.

Name Type Default value Description
enableComputeIsolation

boolean

False

The flag indicates whether enable compute isolation or not.
hostSku

string

The host sku.

ComputeProfile

Describes the compute profile.

Name Type Description
roles

Role[]

The list of roles in the cluster.

ConnectivityEndpoint

The connectivity properties

Name Type Description
location

string

The location of the endpoint.
name

string

The name of the endpoint.
port

integer

The port to connect to.
privateIPAddress

string

The private ip address of the endpoint.
protocol

string

The protocol of the endpoint.

createdByType

The type of identity that created the resource.

Name Type Description
Application

string

Key

string

ManagedIdentity

string

User

string

DataDisksGroups

The data disks groups for the role.

Name Type Description
diskSizeGB

integer

ReadOnly. The DiskSize in GB. Do not set this value.
disksPerNode

integer

The number of disks per node.
storageAccountType

string

ReadOnly. The storage account type. Do not set this value.

DaysOfWeek

Days of the week for a schedule-based autoscale rule

Name Type Description
Friday

string

Monday

string

Saturday

string

Sunday

string

Thursday

string

Tuesday

string

Wednesday

string

DirectoryType

The directory type.

Name Type Description
ActiveDirectory

string

DiskEncryptionProperties

The disk encryption properties

Name Type Default value Description
encryptionAlgorithm

JsonWebKeyEncryptionAlgorithm

Algorithm identifier for encryption, default RSA-OAEP.
encryptionAtHost

boolean

False

Indicates whether or not resource disk encryption is enabled.
keyName

string

Key name that is used for enabling disk encryption.
keyVersion

string

Specific key version that is used for enabling disk encryption.
msiResourceId

string

Resource ID of Managed Identity that is used to access the key vault.
vaultUri

string

Base key vault URI where the customers key is located eg. https://myvault.vault.azure.net

EncryptionInTransitProperties

The encryption-in-transit properties.

Name Type Default value Description
isEncryptionInTransitEnabled

boolean

False

Indicates whether or not inter cluster node communication is encrypted in transit.

ErrorResponse

Describes the format of Error response.

Name Type Description
code

string

Error code
message

string

Error message indicating why the operation failed.

Errors

The error message associated with the cluster creation.

Name Type Description
code

string

The error code.
message

string

The error message.

ExcludedServicesConfig

The configuration that services will be excluded when creating cluster.

Name Type Description
excludedServicesConfigId

string

The config id of excluded services.
excludedServicesList

string

The list of excluded services.

HardwareProfile

The hardware profile.

Name Type Description
vmSize

string

The size of the VM

HDInsightClusterProvisioningState

The provisioning state, which only appears in the response.

Name Type Description
Canceled

string

Deleting

string

Failed

string

InProgress

string

Succeeded

string

IPConfiguration

The ip configurations for the private link service.

Name Type Description
id

string

The private link IP configuration id.
name

string

The name of private link IP configuration.
properties.primary

boolean

Indicates whether this IP configuration is primary for the corresponding NIC.
properties.privateIPAddress

string

The IP address.
properties.privateIPAllocationMethod

PrivateIPAllocationMethod

The method that private IP address is allocated.
properties.provisioningState

PrivateLinkConfigurationProvisioningState

The private link configuration provisioning state, which only appears in the response.
properties.subnet

ResourceId

The subnet resource id.
type

string

The type of the private link IP configuration.

JsonWebKeyEncryptionAlgorithm

Algorithm identifier for encryption, default RSA-OAEP.

Name Type Description
RSA-OAEP

string

RSA-OAEP-256

string

RSA1_5

string

KafkaRestProperties

The kafka rest proxy configuration which contains AAD security group information.

Name Type Description
clientGroupInfo

ClientGroupInfo

The information of AAD security group.
configurationOverride

object

The configurations that need to be overriden.

LinuxOperatingSystemProfile

The ssh username, password, and ssh public key.

Name Type Description
password

string

The password.
sshProfile

SshProfile

The SSH profile.
username

string

The username.

NetworkProperties

The network properties.

Name Type Description
privateLink

PrivateLink

Indicates whether or not private link is enabled.
resourceProviderConnection

ResourceProviderConnection

The direction for the resource provider connection.

OsProfile

The Linux operation systems profile.

Name Type Description
linuxOperatingSystemProfile

LinuxOperatingSystemProfile

The Linux OS profile.

OSType

The type of operating system.

Name Type Description
Linux

string

Windows

string

PrivateEndpoint

The private endpoint.

Name Type Description
id

string

The private endpoint id.

PrivateEndpointConnection

The private endpoint connection.

Name Type Description
id

string

Fully qualified resource ID for the resource. Ex - /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}
name

string

The name of the resource
properties.linkIdentifier

string

The link identifier.
properties.privateEndpoint

PrivateEndpoint

The private endpoint of the private endpoint connection
properties.privateLinkServiceConnectionState

PrivateLinkServiceConnectionState

The private link service connection state.
properties.provisioningState

PrivateEndpointConnectionProvisioningState

The provisioning state, which only appears in the response.
systemData

systemData

Metadata pertaining to creation and last modification of the resource.
type

string

The type of the resource. E.g. "Microsoft.Compute/virtualMachines" or "Microsoft.Storage/storageAccounts"

PrivateEndpointConnectionProvisioningState

The provisioning state, which only appears in the response.

Name Type Description
Canceled

string

Deleting

string

Failed

string

InProgress

string

Succeeded

string

Updating

string

PrivateIPAllocationMethod

The method that private IP address is allocated.

Name Type Description
dynamic

string

static

string

Indicates whether or not private link is enabled.

Name Type Description
Disabled

string

Enabled

string

PrivateLinkConfiguration

The private link configuration.

Name Type Description
id

string

The private link configuration id.
name

string

The name of private link configuration.
properties.groupId

string

The HDInsight private linkable sub-resource name to apply the private link configuration to. For example, 'headnode', 'gateway', 'edgenode'.
properties.ipConfigurations

IPConfiguration[]

The IP configurations for the private link service.
properties.provisioningState

PrivateLinkConfigurationProvisioningState

The private link configuration provisioning state, which only appears in the response.
type

string

The type of the private link configuration.

PrivateLinkConfigurationProvisioningState

The private link configuration provisioning state, which only appears in the response.

Name Type Description
Canceled

string

Deleting

string

Failed

string

InProgress

string

Succeeded

string

PrivateLinkServiceConnectionState

The private link service connection state.

Name Type Description
actionsRequired

string

Whether there is further actions.
description

string

The optional description of the status.
status

PrivateLinkServiceConnectionStatus

The concrete private link service connection.

PrivateLinkServiceConnectionStatus

The concrete private link service connection.

Name Type Description
Approved

string

Pending

string

Rejected

string

Removed

string

QuotaInfo

The quota properties for the cluster.

Name Type Description
coresUsed

integer

The cores used by the cluster.

ResourceId

The azure resource id.

Name Type Description
id

string

The azure resource id.

ResourceIdentityType

The type of identity used for the cluster. The type 'SystemAssigned, UserAssigned' includes both an implicitly created identity and a set of user assigned identities.

Name Type Description
None

string

SystemAssigned

string

SystemAssigned, UserAssigned

string

UserAssigned

string

ResourceProviderConnection

The direction for the resource provider connection.

Name Type Description
Inbound

string

Outbound

string

Role

Describes a role on the cluster.

Name Type Default value Description
VMGroupName

string

The name of the virtual machine group.
autoscale

Autoscale

The autoscale configurations.
dataDisksGroups

DataDisksGroups[]

The data disks groups for the role.
encryptDataDisks

boolean

False

Indicates whether encrypt the data disks.
hardwareProfile

HardwareProfile

The hardware profile.
minInstanceCount

integer

The minimum instance count of the cluster.
name

string

The name of the role.
osProfile

OsProfile

The operating system profile.
scriptActions

ScriptAction[]

The list of script actions on the role.
targetInstanceCount

integer

The instance count of the cluster.
virtualNetworkProfile

VirtualNetworkProfile

The virtual network profile.

ScriptAction

Describes a script action on role on the cluster.

Name Type Description
name

string

The name of the script action.
parameters

string

The parameters for the script provided.
uri

string

The URI to the script.

SecurityProfile

The security profile which contains Ssh public key for the HDInsight cluster.

Name Type Description
aaddsResourceId

string

The resource ID of the user's Azure Active Directory Domain Service.
clusterUsersGroupDNs

string[]

Optional. The Distinguished Names for cluster user groups
directoryType

DirectoryType

The directory type.
domain

string

The organization's active directory domain.
domainUserPassword

string

The domain admin password.
domainUsername

string

The domain user account that will have admin privileges on the cluster.
ldapsUrls

string[]

The LDAPS protocol URLs to communicate with the Active Directory.
msiResourceId

string

User assigned identity that has permissions to read and create cluster-related artifacts in the user's AADDS.
organizationalUnitDN

string

The organizational unit within the Active Directory to place the cluster and service accounts.

SshProfile

The list of SSH public keys.

Name Type Description
publicKeys

SshPublicKey[]

The list of SSH public keys.

SshPublicKey

The SSH public key for the cluster nodes.

Name Type Description
certificateData

string

The certificate for SSH.

StorageAccount

The storage Account.

Name Type Description
container

string

The container in the storage account, only to be specified for WASB storage accounts.
fileSystem

string

The filesystem, only to be specified for Azure Data Lake Storage Gen 2.
fileshare

string

The file share name.
isDefault

boolean

Whether or not the storage account is the default storage account.
key

string

The storage account access key.
msiResourceId

string

The managed identity (MSI) that is allowed to access the storage account, only to be specified for Azure Data Lake Storage Gen 2.
name

string

The name of the storage account.
resourceId

string

The resource ID of storage account, only to be specified for Azure Data Lake Storage Gen 2.
saskey

string

The shared access signature key.

StorageProfile

The storage profile.

Name Type Description
storageaccounts

StorageAccount[]

The list of storage accounts in the cluster.

systemData

Metadata pertaining to creation and last modification of the resource.

Name Type Description
createdAt

string

The timestamp of resource creation (UTC).
createdBy

string

The identity that created the resource.
createdByType

createdByType

The type of identity that created the resource.
lastModifiedAt

string

The timestamp of resource last modification (UTC)
lastModifiedBy

string

The identity that last modified the resource.
lastModifiedByType

createdByType

The type of identity that last modified the resource.

Tier

The cluster tier.

Name Type Description
Premium

string

Standard

string

UserAssignedIdentity

The User Assigned Identity

Name Type Description
clientId

string

The client id of user assigned identity.
principalId

string

The principal id of user assigned identity.
tenantId

string

The tenant id of user assigned identity.

VirtualNetworkProfile

The virtual network properties.

Name Type Description
id

string

The ID of the virtual network.
subnet

string

The name of the subnet.