Azure Data Factory Not Communicating via Private Endpoint

Anon4343 496

After following the instructions to create a Private Endpoint and disable public access to the Azure Data Factory, the self-hosted Integration Runtime is unable to connect. What configuration am I missing?

Instructions followed: https://learn.microsoft.com/en-us/azure/data-factory/data-factory-private-link

Error: The access to data factory from public network is blocked.

The VM can resolve the private link. The private link is associated with Data Factory sub resource not portal. There are no NSGs blocking between the VM VNet and the Private Link VNet.

1 answer

Bhargava-MSFT 31,211 Reputation points Microsoft Employee

2022-11-23T23:53:19.94+00:00
Hello @Anon4343 ,

Welcome to the MS Q&A platform.

The issue is, your SHIR VM is unable to connect to ADF associated with the private endpoint.

You can use one of the following options to configure your DNS settings for private endpoints:

1) Use the host file. You can use the host file on a virtual machine to override the DNS.

2) Use a private DNS zone. You can use private DNS zones to override the DNS resolution for a private endpoint. A private DNS zone can be linked to your virtual network to resolve specific domains.

3) Use your DNS forwarder (optional). You can use your DNS forwarder to override the DNS resolution for a private link resource. Create a DNS forwarding rule to use a private DNS zone on your DNS server hosted in a virtual network.

Reference document: https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns

I hope this helps. Please let me know if you have any further questions.

Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

Want a reminder to come back and check responses? Here is how to subscribe to a notification

If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
Please sign in to rate this answer.

1 person found this answer helpful.
Anon4343 496 Reputation points

2022-11-28T19:41:09.583+00:00

Thanks Bhargava, we are using a Private DNS forwarder and the DNS lookups resolve correctly to the internal IP address.

Bhargava-MSFT 31,211 Reputation points Microsoft Employee

2022-11-28T23:13:38.45+00:00

Hello @Anon4343 ,
Thank you. Were you able to resolve the issue? Please let us know If you have any further questions.

Anon4343 496 Reputation points

2022-11-29T15:00:58.437+00:00

The issue was not resolved. DNS resolution is correct. Is there anything else to check for?

Bhargava-MSFT 31,211 Reputation points Microsoft Employee

2022-11-29T23:30:21.633+00:00

Hello @Anon4343 ,
Can you please try adding the host file entry on the SHIR VM for testing purposes?
If you are still having issues, please let me know. I will check with my network experts.

Anon4343 496 Reputation points

2022-11-30T21:58:32.797+00:00

It still resolves to the same IP address.

Bhargava-MSFT 31,211 Reputation points Microsoft Employee

2022-11-30T23:58:41.767+00:00

Hello @Anon4343 ,
The issue is due to one of the below reasons.

1) Your virtual machine is not within the subnet that is used to run the self-hosted IR(which connects via the private endpoint link).

(or)

2) Issue with DNS configuration:

When you create a private endpoint, the DNS CNAME resource record for the data factory is updated to an alias in a subdomain with the prefix privatelink. By default, we also create a private DNS zone, corresponding to the privatelink subdomain, with the DNS A resource records for the private endpoints.

https://learn.microsoft.com/en-us/azure/data-factory/data-factory-private-link#dns-changes-for-private-endpoints

If you still need further assistance, I suggest filing a support request to look into your configurations further. If you don't have a support plan, I can enable one-time free support for you to work closely on this matter.
I am looking forward to hearing from you.

Anon4343 496 Reputation points

2022-12-01T22:17:58.687+00:00

VNets hosting the VM and Private Endpoints are peered.

The entries in the Private DNS zones appear to be correct.

OK thank you. I'll create a support request and update this posting with the answer.

Bhargava-MSFT 31,211 Reputation points Microsoft Employee

2022-12-02T01:29:06.237+00:00

Thank you so much, @Anon4343

Ben Roberts 0 Reputation points

2025-02-26T04:24:33.79+00:00

What was the solution please?

I have the same issue: DF behind a private endpoint, attempting to register a SHIR from on-prem to it. Connection troubleshooter on the SHIR shows that SHIR is resolving to the correct (private) IP. But getting the same error

Failed to get service token from ADF service with key IR@61eb0e43-75e0-4141-bdf1-cfcb6b06e5f6@[redactedDFName]@ServiceEndpoint=[redactedDFHostName].[redactedRegion].datafactory.azure.net@****************** and time cost is: 0.1606498 seconds, the error code is: InvalidGatewayKey, activityId is: e14a4ef0--f2a3d680435b and detailed error message is Processed HTTP request failed.

Anon4343 496 Reputation points

2025-02-26T14:51:38.1833333+00:00

You may need to update your UDRs.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Azure Data Factory Not Communicating via Private Endpoint

1 answer

Your answer