NSG rule for Azure Data Factory for North Europe Region

Question

HI All,

How we can create a NSG rule for Azure Data Factory for North Europe Region. so we can use selects network when we use Azure Batch Account.

Requirement is connect ADF to Azure Batch to creating NSG rule. please help

Answer 1

You need to configure the NSG and ensure that it allows necessary traffic between ADF and Azure Batch, especially if you're using a virtual network and Azure Private Link for private connections.

First, you need to verify that Azure Data Factory is using a Private Endpoint to connect to services like Azure Batch over a private network. This requires ADF to be deployed within a VNet, and the Private Endpoint must be set up to ensure the communication is routed privately.

Then to set up the NSG for the VNet Subnet:

• Go to the Azure portal, and navigate to your Virtual Network (VNet) where Azure Data Factory and Azure Batch are deployed.
• Select the Subnet that will be used by ADF and Azure Batch.
• Then, add or modify the NSG (Network Security Group) rules associated with this subnet.

To create NSG Rules:

To allow communication between ADF and Azure Batch, add the following Inbound and Outbound rules based on the required IP ranges and ports:

• Inbound Rules:
  • Allow traffic from Azure Data Factory to Azure Batch:
    • Source: AzureLoadBalancer or specify the IP range of ADF's private endpoint.
    • Destination: Any (or the specific IP address/range for the Azure Batch service).
    • Protocol: TCP
    • Port Range: 443 (for HTTPS communication).
  • You may need to allow specific ports for Azure Batch as per its documentation if the default ports do not suffice.
• Outbound Rules:
  • Allow outbound traffic from Azure Data Factory to Azure Batch:
    • Source: Any (or the specific subnet that ADF resides in).
    • Destination: The IP address or private endpoint of Azure Batch in the North Europe region.
    • Protocol: TCP
    • Port Range: 443.

You may also want to use Service Tags for common Azure services (like AzureBatch and AzureDataFactory) instead of specific IPs to avoid having to manually update rules if IPs change:

Add rules for AzureBatch and AzureDataFactory service tags as destinations, which helps simplify maintenance.

Let me know if you need further clarification or specific examples for your configuration!