設定 Azure HDInsight 的虛擬網路服務端點原則
本文提供如何使用 Azure HDInsight 在虛擬網路上實作服務端點原則的相關資訊。
背景
Azure HDInsight 可讓您在自己的虛擬網路中建立叢集。 如果您需要允許從虛擬網路到其他 Azure 服務 (例如儲存體帳戶) 的傳出流量,您可以建立服務端點原則。 不過,透過 Azure 入口網站建立的服務端點原則,只允許您為單一帳戶、訂用帳戶中的所有帳戶或資源群組中的所有帳戶建立原則。
不過,作為受控服務,Azure HDInsight 會從每個區域中特定儲存體帳戶中的每個叢集收集資料和記錄檔。 為了讓此資料從虛擬網路觸達 HDInsight,您必須建立服務端點原則,以允許傳出流量流向 Azure HDInsight 所管理的特定資料收集點。
HDInsight 的服務端點原則
這些服務端點原則支援下列功能:
- 叢集建立、作業執行和平台作業 (例如調整) 的記錄和遙測集合。
- 將虛擬硬碟 (VHD) 連結至新建立的叢集節點,以便在叢集上佈建軟體和程式庫。
如果未建立服務端點原則來啟用此資料流程,叢集建立可能會失敗,且 Azure HDInsight 將無法為您的叢集提供支援。
建立 HDInsight 的服務端點原則
建立新的叢集之前,請確定已將正確的服務端點原則連結至您的虛擬網路。 否則,叢集建立可能會失敗或造成錯誤。
使用下列程序來建立必要的服務端點原則:
決定您要建立 HDInsight 叢集的區域。
在服務端點原則資源清單中查閱該區域,這會為 HDInsight 管理儲存體帳戶提供所有資源群組。
選取您區域的資源群組清單。
Canada Central
的資源範例如下所示:"Canada Central":[ "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/Default-Storage-WestUS", "/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f/resourceGroups/GenevaWarmPathManageRG", "/subscriptions/cccc2c2c-dd3d-ee4e-ff5f-aaaaaa6a6a6a/resourceGroups/GenevaWarmPathManageRG", "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/Default-Storage-CanadaCentral", "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/cancstorage", "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/GenevaWarmPathManageRG", "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/hdi31distrorelease", "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/bigdatadistro" ],
將資源群組清單插入以 Azure CLI 或 Azure PowerShell 撰寫的安裝指令碼中。
$subscriptionId = "<subscription id>" $rgName="<resource group name> " $location="<location name>" $vnetName="<vnet name>" $subnetName="<subnet name>" $sepName="<service endpoint policy name>" $sepDefName="<service endpoint policy definition name>" # Set to the right subscription ID az account set --subscription $subscriptionId # setup service endpoint on the virtual network subnet az network vnet subnet update -g $rgName --vnet-name $vnetName -n $subnetName --service-endpoints Microsoft.Storage # Create Service Endpoint Policy az network service-endpoint policy create -g $rgName -n $sepName -l $location # Insert the list of HDInsight owned resources for the region your clusters will be created in. # Be sure to get the most recent list of resource groups from the [list of service endpoint policy resources](https://github.com/Azure-Samples/hdinsight-enterprise-security/blob/main/hdinsight-service-endpoint-policy-resources.json) [String[]]$resources = @("/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/Default-Storage-WestUS",` "/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f/resourceGroups/GenevaWarmPathManageRG",` "/subscriptions/cccc2c2c-dd3d-ee4e-ff5f-aaaaaa6a6a6a/resourceGroups/GenevaWarmPathManageRG",` "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/Default-Storage-CanadaCentral",` "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/cancstorage",` "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/GenevaWarmPathManageRG", "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/hdi31distrorelease", "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/bigdatadistro") #Assign service resources to the SEP policy. az network service-endpoint policy-definition create -g $rgName --policy-name $sepName -n $sepDefName --service "Microsoft.Storage" --service-resources $resources # Associate a subnet to the service endpoint policy just created. If there is a delay in updating it to subnet, you can use the Azure portal to associate the policy with the subnet. az network vnet subnet update -g $rgName --vnet-name $vnetName -n $subnetName --service-endpoint-policy $sepName
如果您想要使用 PowerShell 設定服務端點原則,請使用下列程式碼片段。
#Script to assign SEP $subscriptionId = "<subscription id>" $rgName = "<resource group name>" $vnetName = "<vnet name>" $subnetName = "<subnet Name" $location = "Canada Central" # Connect to your Azure Account Connect-AzAccount # Select the Subscription that you want to use Select-AzSubscription -SubscriptionId $subscriptionId # Retrieve VNet Config $vnet = Get-AzVirtualNetwork -ResourceGroupName $rgName -Name $vnetName # Retrieve Subnet Config $subnet = Get-AzVirtualNetworkSubnetConfig -Name $subnetName -VirtualNetwork $vnet # Insert the list of HDInsight owned resources for the region your clusters will be created in. # Be sure to get the most recent list of resource groups from the [list of service endpoint policy resources](https://github.com/Azure-Samples/hdinsight-enterprise-security/blob/main/hdinsight-service-endpoint-policy-resources.json) [String[]]$resources = @("/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/Default-Storage-WestUS", "/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f/resourceGroups/GenevaWarmPathManageRG", "/subscriptions/cccc2c2c-dd3d-ee4e-ff5f-aaaaaa6a6a6a/resourceGroups/GenevaWarmPathManageRG", "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/Default-Storage-CanadaCentral", "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/cancstorage", "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/GenevaWarmPathManageRG", "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/hdi31distrorelease", "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/bigdatadistro") #Declare service endpoint policy definition $sepDef = New-AzServiceEndpointPolicyDefinition -Name "SEPHDICanadaCentral" -Description "Service Endpoint Policy Definition" -Service "Microsoft.Storage" -ServiceResource $resources # Service Endpoint Policy $sep= New-AzServiceEndpointPolicy -ResourceGroupName $rgName -Name "SEPHDICanadaCentral" -Location $location -ServiceEndpointPolicyDefinition $sepDef # Associate a subnet to the service endpoint policy just created. If there is a delay in updating it to subnet, you can use the Azure portal to associate the policy with the subnet. Set-AzVirtualNetworkSubnetConfig -Name $subnetName -VirtualNetwork $vnet -AddressPrefix $subnet.AddressPrefix -ServiceEndpointPolicy $sep
重要
建議您以手動方式或經由自動化,依照排程取得最新的服務端點原則資源清單。 這可防止在 JSON 檔案中新增或移除其他資源群組時發生 CRUD 問題。