通过

你当前正在访问 Microsoft Azure Global Edition 技术文档网站。 如果需要访问由世纪互联运营的 Microsoft Azure 中国技术文档网站，请访问 https://docs.azure.cn。

为 Azure HDInsight 配置虚拟网络服务终结点策略

  • 项目

本文介绍如何使用 Azure HDInsight 在虚拟网络上实现服务终结点策略。

背景

Azure HDInsight 允许你在自己的虚拟网络中创建群集。 如果需要允许来自虚拟网络的流量传出到其他 Azure 服务（如存储帐户），则可以创建 服务终结点策略。 但是，通过 Azure 门户创建的服务终结点策略仅允许你为单个帐户、订阅中的所有帐户或资源组中的所有帐户创建策略。

但作为一项托管服务，Azure HDInsight 会从每个区域中特定存储帐户中的每个群集收集数据和日志文件。 为了使此数据从虚拟网络访问 HDInsight，必须创建可允许流量传出到由 Azure HDInsight 管理的特定数据收集点的服务终结点策略。

适用于 HDInsight 的服务终结点策略

这些服务终结点策略支持以下功能：

  • 收集有关群集创建、作业执行以及平台操作（例如缩放）的日志和遥测。
  • 将虚拟硬盘 (VHD) 附加到新创建的群集节点，以在群集上预配软件和库。

如果未创建服务终结点策略来启用此数据流，则群集创建可能会失败，且 Azure HDInsight 将无法为群集提供支持。

为 HDInsight 创建服务终结点策略

在创建新群集之前，请确保已将正确的服务终结点策略附加到虚拟网络。 否则，群集创建可能会失败或导致错误。

使用以下过程创建所需的服务终结点策略：

  1. 确定要在其中创建 HDInsight 群集的区域。

  2. 可在服务终结点策略资源列表中查找该区域，其中提供了 HDInsight 管理存储帐户的所有资源组。

  3. 选择区域的资源组列表。 下面为 Canada Central 的资源示例：

    "Canada Central":[
    "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/Default-Storage-WestUS",
    "/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f/resourceGroups/GenevaWarmPathManageRG",
    "/subscriptions/cccc2c2c-dd3d-ee4e-ff5f-aaaaaa6a6a6a/resourceGroups/GenevaWarmPathManageRG",
    "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/Default-Storage-CanadaCentral",
    "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/cancstorage",
    "/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/GenevaWarmPathManageRG",
    "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/hdi31distrorelease",
    "/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/bigdatadistro"
],

  4. 将这列资源组插入用 Azure CLI 或 Azure PowerShell 编写的安装脚本中。

    $subscriptionId = "<subscription id>"
$rgName="<resource group name> "
$location="<location name>"
$vnetName="<vnet name>"
$subnetName="<subnet name>"
$sepName="<service endpoint policy name>"
$sepDefName="<service endpoint policy definition name>"

# Set to the right subscription ID
az account set --subscription $subscriptionId

# setup service endpoint on the virtual network subnet
az network vnet subnet update -g $rgName --vnet-name $vnetName -n $subnetName --service-endpoints Microsoft.Storage

# Create Service Endpoint Policy
az network service-endpoint policy create -g $rgName  -n $sepName -l $location

# Insert the list of HDInsight owned resources for the region your clusters will be created in.
# Be sure to get the most recent list of resource groups from the [list of service endpoint policy resources](https://github.com/Azure-Samples/hdinsight-enterprise-security/blob/main/hdinsight-service-endpoint-policy-resources.json)
[String[]]$resources = @("/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/Default-Storage-WestUS",`
"/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f/resourceGroups/GenevaWarmPathManageRG",`
"/subscriptions/cccc2c2c-dd3d-ee4e-ff5f-aaaaaa6a6a6a/resourceGroups/GenevaWarmPathManageRG",`
"/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/Default-Storage-CanadaCentral",`
"/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/cancstorage",`
"/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/GenevaWarmPathManageRG",
"/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/hdi31distrorelease",
"/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/bigdatadistro")

#Assign service resources to the SEP policy.
az network service-endpoint policy-definition create -g $rgName --policy-name $sepName -n $sepDefName --service "Microsoft.Storage" --service-resources $resources

# Associate a subnet to the service endpoint policy just created. If there is a delay in updating it to subnet, you can use the Azure portal to associate the policy with the subnet.
az network vnet subnet update -g $rgName --vnet-name $vnetName -n $subnetName --service-endpoint-policy $sepName

    如果希望使用 PowerShell 设置服务终结点策略，请使用以下代码片段。

    #Script to assign SEP 
$subscriptionId = "<subscription id>"
$rgName = "<resource group name>"
$vnetName = "<vnet name>"
$subnetName = "<subnet Name"
$location = "Canada Central"

# Connect to your Azure Account
Connect-AzAccount

# Select the Subscription that you want to use
Select-AzSubscription -SubscriptionId $subscriptionId

# Retrieve VNet Config
$vnet = Get-AzVirtualNetwork -ResourceGroupName $rgName -Name $vnetName

# Retrieve Subnet Config
$subnet = Get-AzVirtualNetworkSubnetConfig -Name $subnetName -VirtualNetwork $vnet

# Insert the list of HDInsight owned resources for the region your clusters will be created in.
# Be sure to get the most recent list of resource groups from the [list of service endpoint policy resources](https://github.com/Azure-Samples/hdinsight-enterprise-security/blob/main/hdinsight-service-endpoint-policy-resources.json)
[String[]]$resources = @("/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/Default-Storage-WestUS",
"/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f/resourceGroups/GenevaWarmPathManageRG",
"/subscriptions/cccc2c2c-dd3d-ee4e-ff5f-aaaaaa6a6a6a/resourceGroups/GenevaWarmPathManageRG",
"/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/Default-Storage-CanadaCentral",
"/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/cancstorage",
"/subscriptions/dddd3d3d-ee4e-ff5f-aa6a-bbbbbb7b7b7b/resourceGroups/GenevaWarmPathManageRG",
"/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/hdi31distrorelease",
"/subscriptions/eeee4efe-ff5f-aa6a-bb7b-cccccc8c8c8c/resourceGroups/DistroStorageRG/providers/Microsoft.Storage/storageAccounts/bigdatadistro")

#Declare service endpoint policy definition
$sepDef = New-AzServiceEndpointPolicyDefinition -Name "SEPHDICanadaCentral" -Description "Service Endpoint Policy Definition" -Service "Microsoft.Storage" -ServiceResource $resources

# Service Endpoint Policy
$sep= New-AzServiceEndpointPolicy -ResourceGroupName $rgName -Name "SEPHDICanadaCentral" -Location $location -ServiceEndpointPolicyDefinition $sepDef

# Associate a subnet to the service endpoint policy just created. If there is a delay in updating it to subnet, you can use the Azure portal to associate the policy with the subnet.
Set-AzVirtualNetworkSubnetConfig -Name $subnetName -VirtualNetwork $vnet -AddressPrefix $subnet.AddressPrefix -ServiceEndpointPolicy $sep

重要

建议你定期手动或通过自动化获取最新的服务终结点策略资源列表。 这将防止在 JSON 文件中添加或删除其他资源组时出现 CRUD 问题。

后续步骤