Create a managed private endpoint for Azure Data Explorer

Managed private endpoints are required to connect to Azure resources that are highly protected. They're one-way private connections that allow Azure Data Explorer to connect to other protected services. In this article, you'll learn how to create a managed private endpoint and connect it to your data source.

Prerequisites

• An Azure subscription. Create a free Azure account.
• An Azure Data Explorer cluster that isn't injected in a virtual network.
• An event hub or a Azure Storage blob in a subscription that is registered to the Microsoft.Network resource provider. For more information, see Register subscription to resource provider.

Create a managed private endpoint using the Azure portal

You can create a managed private endpoint using the portal for your cluster to use when accessing your storage.

1. In the Azure portal, navigate to your cluster and then select Networking.

2. Select Managed private endpoints, and then select Add.

   

3. On the New managed private endpoint pane, fill out the resource details with the following information, and then select Next.

   

   Setting	Suggested value	Field description
   Name	mpeToStorage	The name of the managed private endpoint
   Subscription	Your subscription	Select the Azure subscription that you want to use for your cluster
   Resource type	Microsoft.Storage/storageAccounts	Select the relevant resources type you want for your data source.
   Resource name	share	Choose the cluster that should be used as the destination for the new Azure Private Endpoint
   Target sub-resource	blob	Select the relevant target for your data source.

4. select Create to create the managed private endpoint resource.

Create a managed private endpoint using the REST API

Creating a managed private endpoint requires a single API call to the Kusto resource provider. You can establish a managed private endpoint to the following resource types:

• Microsoft.Storage/storageAccounts (sub-resource can be "blob" or "dfs")
• Microsoft.EventHub/namespaces (sub-resource "namespace")
• Microsoft.Devices/IoTHubs (sub-resource "iotHub")
• Microsoft.KeyVault/vaults (sub-resource "vault")
• Microsoft.Sql/servers (sub-resource "sqlServer")
• Microsoft.Kusto/clusters (sub-resource "cluster")
• Microsoft.DigitalTwins/digitalTwinsInstance (sub-resource "digitaltwinsinstance")

In the following this example, you'll use the ARMclient in PowerShell to create a managed private endpoint using the REST API.

Note

Connecting to a storage account a "dfs" resource requires an additional managed private endpoint to the "blob" sub-resource.

Prerequisites for using the REST API

1. Install choco

2. Install ARMClient

   choco install armclient
   

3. Log in with ARMClient

   armclient login
   

Create a managed private endpoint to Azure Event Hubs

Use the following REST API call to enable the managed private endpoint to an Event Hubs service:

1. Run the following command to create a managed private endpoint to an Event Hubs service:

   # Replace the <...> placeholders with the correct values
   armclient PUT /subscriptions/<subscriptionIdADX>/resourceGroups/<resourceGroupNameADX>/providers/Microsoft.Kusto/clusters/<clusterName>/managedPrivateEndpoints/<newMpeName>?api-version=2022-02-01 @"
   {
       'properties': {
           'privateLinkResourceId':'/subscriptions/<subscriptionIdEventHub>/resourceGroups/<resourceGroupNameEventHub>/providers/Microsoft.EventHub/namespaces/<EventHubNamespace>',
           'groupId':'namespace',
           'requestMessage':'Please Approve.'
       }
   }
   "@
   

2. Check the response.

   {
     "id": "/subscriptions/<subscriptionIdADX>/resourceGroups/<resourceGroupNameADX>/providers/Microsoft.Kusto/Clusters/<clusterName>/ManagedPrivateEndpoints/<newMpeName>",
     "name": "<clusterName>/<newMpeName>",
     "type": "Microsoft.Kusto/Clusters/ManagedPrivateEndpoints",
     "location": "DummyLocation",
     "properties": {
       "privateLinkResourceId": "/subscriptions/<subscriptionIdEventHub>/resourceGroups/<resourceGroupNameEventHub>/providers/Microsoft.EventHub/namespaces/<EventHubNamespace>",
       "groupId": "namespace",
       "requestMessage": "Please Approve.",
       "provisioningState": "Creating"
     }
   }
   

Create a managed private endpoint to an Azure Storage account

Use the following REST API call to enable the managed private endpoint to an Azure Storage blob:

1. Run the following command to create a managed private endpoint to Event Hubs:

   #replace the <...> placeholders with the correct values
   armclient PUT /subscriptions/<subscriptionIdADX>/resourceGroups/<resourceGroupNameADX>/providers/Microsoft.Kusto/clusters/<clusterName>/managedPrivateEndpoints/<newMpeName>?api-version=2022-02-01 @"
   {
       'properties': {
           'privateLinkResourceId':'/subscriptions/<subscriptionIdStorage>/resourceGroups/<resourceGroupNameStorage>/providers/Microsoft.Storage/storageAccounts/<storageAccountName>',
           'groupId':'blob',
           'requestMessage':'Please Approve.'
       }
   }
   "@
   

2. Check the response.

   {
     "id": "/subscriptions/<subscriptionIdADX>/resourceGroups/<resourceGroupNameADX>/providers/Microsoft.Kusto/Clusters/<clusterName>/ManagedPrivateEndpoints/<newMpeName>",
     "name": "<clusterName>/<newMpeName>",
     "type": "Microsoft.Kusto/Clusters/ManagedPrivateEndpoints",
     "location": "DummyLocation",
     "properties": {
       "privateLinkResourceId": "/subscriptions/<subscriptionIdStorage>/resourceGroups/<resourceGroupNameStorage>/providers/Microsoft.Storage/storageAccounts/<storageAccountName>",
       "groupId": "blob",
       "requestMessage": "Please Approve.",
       "provisioningState": "Creating"
     }
   }
   

How to check progress

To check the progress of the managed private endpoint migration, use the following command:

1. Run the following command:

   #replace the <...> placeholders with the correct values
   armclient GET /subscriptions/<subscriptionIdADX>/resourceGroups/<resourceGroupNameADX>/providers/Microsoft.Kusto/clusters/<clusterName>/managedPrivateEndpoints/<newMpeName>?api-version=2022-02-01
   

2. Check the response.

   {
     "id": "/subscriptions/<subscriptionIdADX>/resourceGroups/<resourceGroupNameADX>/providers/Microsoft.Kusto/Clusters/<clusterName>/ManagedPrivateEndpoints/<newMpeName>",
     "name": "<clusterName>/<newMpeName>",
     "type": "Microsoft.Kusto/Clusters/ManagedPrivateEndpoints",
     "location": "DummyLocation",
     "properties": {
       "privateLinkResourceId": "/subscriptions/02de0e00-8c52-405c-9088-1342de78293d/resourceGroups/<resourceGroupNameADX>/providers/Microsoft.<service>/<...>/<name>",
       "groupId": "<groupId>",
       "requestMessage": "Please Approve.",
       "provisioningState": "Succeeded"
     },
     "systemData": {
       "createdBy": "<UserName>",
       "createdByType": "User",
       "createdAt": "2022-02-05T08:29:54.2912851Z",
       "lastModifiedBy": "chrisqpublic@contoso.com",
       "lastModifiedByType": "User",
       "lastModifiedAt": "2022-02-05T08:29:54.2912851Z"
     }
   }
   

Approve the managed private endpoint

Whichever method you used to create the managed private endpoint, you must approve its creation on the target resource. To approve a managed private endpoint to an Event Hubs service:

1. In the Azure portal, navigate to your Event Hubs service and then select Networking.

2. Select Private endpoint connections, select the managed private endpoint you created, and then select Approve.

   

3. In the Connection state column, verify that the managed private endpoint is approved.

   

Your cluster can now connect to the resource using the managed private endpoint connection.

Create multiple managed private endpoints

You can create multiple managed private endpoints using ARM templates and Terraform. The following examples ensure that the managed private endpoint to the Event Hubs namespace is created before the one to the Storage account.

ARM template

The following example uses an ARM template to create two managed private endpoints in an Azure Data Explorer cluster. The first endpoint connects to an Event Hubs namespace. The second endpoint connects to a Storage account, with a dependency that ensures that the Event Hubs endpoint is created first.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "cluster_name": {
            "defaultValue": "<ADX cluster name>",
            "type": "String"
        },
        "eventhub_resource_id": {
            "defaultValue": "<Eventhub resource id>",
            "type": "String"
        },
        "storage_resource_id": {
            "defaultValue": "<Storage resource id>",
            "type": "String"
        },
        "managed_pe_eventhub_name": {
            "defaultValue": "<name of the managed private endpoint to Event Hub>",
            "type": "String"
        },
        "managed_pe_storage_name": {
            "defaultValue": "<name of the managed private endpoint to Storage>",
            "type": "String"
        }
    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.Kusto/Clusters",
            "apiVersion": "2023-08-15",
            "name": "[parameters('cluster_name')]",
            "location": "<region of the cluster>",
            "sku": {...},
            "zones": {...}
            "properties": {...}
        },
        {
            "type": "Microsoft.Kusto/Clusters/ManagedPrivateEndpoints",
            "apiVersion": "2023-08-15",
            "name": "[concat(parameters('cluster_name'), '/', parameters('managed_pe_eventhub_name'))]",
            "dependsOn": [
                "[resourceId('Microsoft.Kusto/Clusters', parameters('cluster_name'))]"
            ],
            "properties": {
                "privateLinkResourceId": "[parameters('eventhub_resource_id')]",
                "groupId": "namespace",
                "requestMessage": "Please approve"
            }
        },
        {
            "type": "Microsoft.Kusto/Clusters/ManagedPrivateEndpoints",
            "apiVersion": "2023-08-15",
            "name": "[concat(parameters('cluster_name'), '/', parameters('managed_pe_storage_name'))]",
            "dependsOn": [
                "[resourceId('Microsoft.Kusto/Clusters', parameters('cluster_name'))]",
                "[resourceId('Microsoft.Kusto/Clusters/ManagedPrivateEndpoints', parameters('cluster_name'), parameters('managed_pe_eventhub_name'))]"
            ],
            "properties": {
                "privateLinkResourceId": "[parameters('storage_resource_id')]",
                "groupId": "blob",
                "requestMessage": "Please approve"
            }
        }
    ]
}

Terraform configuration

The following example uses a Terraform configuration that creates two managed private endpoints in an Azure Data Explorer cluster. The first endpoint connects to an Event Hubs namespace. The second endpoint connects to a Storage account, with a dependency that ensures that the Event Hubs endpoint is created first.

resource "azapi_resource" "mpe_to_eventhub" {
  type = "Microsoft.Kusto/clusters/managedPrivateEndpoints@2023-08-15"
  name                 = "mpeToEventHub"
  parent_id            = "<the resource id of the cluster>"
  body = jsonencode({
    properties = {
      groupId = "namespace"
      privateLinkResourceId = "<The ARM resource ID of the EventHub for which the managed private endpoint is created.>"
      requestMessage = "Please Approve."
    }
  })
}

resource "azapi_resource" "mpe_to_storage" {
  type = "Microsoft.Kusto/clusters/managedPrivateEndpoints@2023-08-15"
  name                 = "mpeToStorage"
  parent_id            = "<the resource id of the cluster>"
  body = jsonencode({
    properties = {
      groupId = "blob"
      privateLinkResourceId = "<The ARM resource ID of the Storage Account for which the managed private endpoint is created.>"
      requestMessage = "Please Approve."
    }
  })
  depends_on = [
    azapi_resource.mpe_to_eventhub
  ]
}

Automatic approval

You can automatically approve a managed private endpoint if the requesting identity has the Microsoft.<Provider>/<ResourceType>/privateEndpointConnectionsApproval/action permission on the target resource of the managed private endpoint.

Related content

• Troubleshooting private endpoints in Azure Data Explorer