Spark HDInsight cluster deployment fails with an internal server error

Michael P Spertus 0 Reputation points
2024-10-27T20:54:05.21+00:00

I've tried deploying a 5.1.3 HDInsight cluster twice, and both times it fails with an internal server error. JSON below. I chose all the defaults except for using an existing storage account and virtual network. Any suggestions appreciated.

{
    "authorization": {
        "action": "Microsoft.HDInsight/clusters/write",
        "scope": "/subscriptions/b2c3e770-2991-4cba-a0ff-2a4dea242464/resourcegroups/MPCS53014RG/providers/Microsoft.HDInsight/clusters/spark3-mpcs53014-2024"
    },
    "caller": "spertus@UCHICAGO.EDU",
    "channels": "Operation",
    "claims": {
        "aud": "https://management.core.windows.net/",
        "iss": "https://sts.windows.net/83b02c92-5f26-48ed-9e5b-6c2fca46a8e6/",
        "iat": "1730060096",
        "nbf": "1730060096",
        "exp": "1730064124",
        "http://schemas.microsoft.com/claims/authnclassreference": "1",
        "aio": "ATQAy/8YAAAAUKLFhA4ZasItJuy26CL1TGtV+E04Chs8X5lazPznHhvM8sVXvBQxhFUjTpSAj1GQ",
        "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd",
        "appid": "c44b4083-3bb0-49c1-b47d-974e53cbdf3c",
        "appidacr": "2",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname": "Spertus",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname": "Michael",
        "groups": "fba39e04-af97-4aba-8b98-f7894e0e8b22,ffa5e605-b678-4861-8bdf-c94f1921a4c5,d82f0111-d9fa-4edf-8bce-dd11ac102c87,41a30c2d-0c49-4b8e-b23c-74f64864256c,c6e50850-fe2f-4bf9-8b34-a8195152ce6f,496d4d73-f18d-43ae-a0d5-6188e69efac3,6d210f7d-0fd8-4866-8bb4-110fb41d2124,1b40a380-1d81-41a2-abb4-c34ead2485ea,b4c3fd81-7a1e-4ca4-9818-706b5b3a019f,23f2a48e-0dd9-4d27-9016-3347b88eacd1,82a3e095-049b-48f3-b7dd-06f806c2827d,1e7ba596-05e0-43a4-997e-72c213f2b4fc,86d0dbae-9ea2-4fcf-a981-eda64c2feab4,78ee38b4-f158-443a-900b-58eb5b7e8f0f,ba43cbc0-7527-4519-ab11-2ec4bc4dd1b7,8ff4dac9-ae12-4d2e-b5d3-ff6ac6b93406,980c47ff-613a-4805-8b2d-fa53c15a4ed7",
        "idtyp": "user",
        "ipaddr": "24.148.80.78",
        "name": "Michael P Spertus",
        "http://schemas.microsoft.com/identity/claims/objectidentifier": "491208eb-696b-4f59-b504-a33bcb645f48",
        "onprem_sid": "S-1-5-21-1644491937-1604221776-725345543-92611",
        "puid": "1003BFFD9FB092E3",
        "rh": "0.AXYAkiywgyZf7UieW2wvykao5kZIf3kAutdPukPawfj2MBO0ABk.",
        "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "eOYRz-GY2k8IuNOxrw3vrFP88htF443pvMIbJ1octlE",
        "http://schemas.microsoft.com/identity/claims/tenantid": "83b02c92-5f26-48ed-9e5b-6c2fca46a8e6",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "spertus@UCHICAGO.EDU",
        "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn": "spertus@UCHICAGO.EDU",
        "uti": "6QZjzVLVQEastazsP88IAA",
        "ver": "1.0",
        "xms_idrel": "1 16",
        "xms_tcdt": "1450452263"
    },
    "correlationId": "4eb1c99d-3823-4135-bc6d-430da2231163",
    "description": "",
    "eventDataId": "b931372c-cf63-4d41-8f20-363e595e9928",
    "eventName": {
        "value": "EndRequest",
        "localizedValue": "End request"
    },
    "category": {
        "value": "Administrative",
        "localizedValue": "Administrative"
    },
    "eventTimestamp": "2024-10-27T20:40:27.3411586Z",
    "id": "/subscriptions/b2c3e770-2991-4cba-a0ff-2a4dea242464/resourcegroups/MPCS53014RG/providers/Microsoft.HDInsight/clusters/spark3-mpcs53014-2024/events/b931372c-cf63-4d41-8f20-363e595e9928/ticks/638656584273411586",
    "level": "Error",
    "operationId": "2ecb6285-6265-4515-b1f0-e16c2d7c5c2b",
    "operationName": {
        "value": "Microsoft.HDInsight/clusters/write",
        "localizedValue": "Create or Update Cluster"
    },
    "resourceGroupName": "MPCS53014RG",
    "resourceProviderName": {
        "value": "Microsoft.HDInsight",
        "localizedValue": "Microsoft.HDInsight"
    },
    "resourceType": {
        "value": "Microsoft.HDInsight/clusters",
        "localizedValue": "Microsoft.HDInsight/clusters"
    },
    "resourceId": "/subscriptions/b2c3e770-2991-4cba-a0ff-2a4dea242464/resourcegroups/MPCS53014RG/providers/Microsoft.HDInsight/clusters/spark3-mpcs53014-2024",
    "status": {
        "value": "Failed",
        "localizedValue": "Failed"
    },
    "subStatus": {
        "value": "",
        "localizedValue": ""
    },
    "submissionTimestamp": "2024-10-27T20:41:32Z",
    "subscriptionId": "b2c3e770-2991-4cba-a0ff-2a4dea242464",
    "tenantId": "83b02c92-5f26-48ed-9e5b-6c2fca46a8e6",
    "properties": {
        "statusMessage": "{\"status\":\"Failed\",\"error\":{\"code\":\"ResourceOperationFailure\",\"message\":\"The resource operation completed with terminal provisioning state 'Failed'.\",\"details\":[{\"code\":\"InternalServerError\",\"message\":\"Internal server error occurred while processing the request. Please retry the request or contact support.\"}]}}",
        "eventCategory": "Administrative",
        "entity": "/subscriptions/b2c3e770-2991-4cba-a0ff-2a4dea242464/resourcegroups/MPCS53014RG/providers/Microsoft.HDInsight/clusters/spark3-mpcs53014-2024",
        "message": "Microsoft.HDInsight/clusters/write",
        "hierarchy": "83b02c92-5f26-48ed-9e5b-6c2fca46a8e6/Production-Management-Group/b2c3e770-2991-4cba-a0ff-2a4dea242464"
    },
    "relatedEvents": []
}
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
212 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Smaran Thoomu 16,735 Reputation points Microsoft Vendor
    2024-10-28T10:25:15.28+00:00

    Hi @Michael P Spertus
    Thank you for reaching out, and I understand the frustration of encountering repeated internal server errors when deploying the HDInsight cluster.

    Given that you’re using a custom storage account and virtual network, I suggest the following steps to help troubleshoot:

    1. Verify that the storage account and virtual network configurations are correctly set up with appropriate permissions for HDInsight. Sometimes access-related issues can lead to unexpected errors during deployment.
    2. Occasionally, region-based quota limits or temporary issues in the Azure region might cause deployment failures. Ensure that the resource group’s region supports the HDInsight cluster configuration you’re attempting.
    3. You could attempt deploying through the Azure CLI or PowerShell to see if a different method yields more insights or error details.

    If the issue persists after these checks, it may be due to an underlying issue with the HDInsight service. In that case, I would suggest you raise the support ticket for further investigation.

    I hope this helps. Please let me know if you have any questions.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.