Greetings, I would appreciate assistance with a design to a solution I would like to implement on Azure. Below I summarized all the information and approaches I tried. Customer has a private endpoint on a "consumer" VNet, from which I would like to allow them to privately consume a service. The private endpoint is to be connected to a private-link resource in a "producer-surrogate" VNet. The aforementioned private-link resource will route traffic to a VPN gateway (also contained in the "producer-surrogate" VNet). Said VPN gateway is connected to a remote service VNet hosting the remote service. I'm allowed to commit changes only to resources in the "producer-surrogate" VNet, i.e., the private endpoint on the "consumer" VNet is the only way for the consumer to consume. Connected to the above private endpoint I've looked into: A private link service, but those are not supporting a Standard Load Balancer with a backend pool configured by an IP address, hence it cannot be used to route traffic to a VPN gateway. I am aware that routing traffic to a VM, followed by a user defined route, directing traffic to the VPN gateway is possible, but I am not interested in a solution I would have to maintain and scale myself. An application gateway connected to a standard Azure Firewall with forced tunneling (which is used to route traffic to the VPN gateway) should work, however, this seems to be an extremely high cost solution for a simple routing job. I will appreciate your expertise in the matter, Thanks.

@Omer Cohen , Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well. From your verbatim, The VNET "consumer" has a Private EndPoint And this VNET can use this PE only to access a resource in a remote Network which is connected to the VNET "producer-surrogate" via VPN Gateway. Observation, I am afraid I do not see the requirement for using a Azure Firewall with forced tunneling in this case. May I ask if it is not possible to directly add the IP Address or FQDN of the "resource" as the backend Pool See : App Gw Components Backend pools You don't need the Azure Firewall to do a forced tunneling See : Application Gateway Private Link Kindly let us know if this helps or you need further assistance on this issue. Thanks, Kapil Please don’t forget to close the thread by clicking " Accept the answer " wherever the information provided helps you, as this can be beneficial to other community members.

Exploring Cost-Effective Solutions for Routing Traffic from an Azure Private Endpoint to a VPN Gateway

Accepted answer

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-11-04T05:28:18.7666667+00:00
@Omer Cohen ,

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

From your verbatim,

The VNET "consumer" has a Private EndPoint

And this VNET can use this PE only to access a resource in a remote Network which is connected to the VNET "producer-surrogate" via VPN Gateway.

Observation,

I am afraid I do not see the requirement for using a Azure Firewall with forced tunneling in this case.

May I ask if it is not possible to directly add the IP Address or FQDN of the "resource" as the backend Pool

See : App Gw Components Backend pools

You don't need the Azure Firewall to do a forced tunneling

See : Application Gateway Private Link

Kindly let us know if this helps or you need further assistance on this issue.

Thanks,

Kapil

Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.
Please sign in to rate this answer.

1 person found this answer helpful.
Omer Cohen 40 Reputation points

2024-11-04T22:38:21.8133333+00:00

Hey @KapilAnanth-MSFT ,
I appreciate you taking the time to go over my issue in depth.
If i understand correctly, I can use the private IP of a VPN gateway as the backend member of an application gateway, to enable the following traffic flow:
Private-endpoint (on VNet 'A') -> Application Gateway (on VNet 'B') -> internal IP of a VPN gateway (on VNet 'B'), correct?
I assumed it wouldn't be possible due to the different OSI layers the Application and VPN gateways operate on.
When I did try to add an internal IP of a VPN gateway as a backend member, it was marked as unhealthy by the default health probe.

Many thanks,
Omer.

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-11-05T05:06:44.13+00:00

@Omer Cohen ,

You are welcome.

Wrt, "If i understand correctly, I can use the private IP of a VPN gateway as the backend member of an application gateway, to enable the following traffic flow:"

No, this is incorrect

You should use the private IP/FQDN of your "resource" which is accessed over the VPN Gateway as the backend member of the application gateway

Not the VPN Gateway itself

The flow should be,

Private-endpoint (on VNet 'A') -> Application Gateway (on VNet 'B' via PLS) -> internal IP/FQDN of the resource to which you want to connect to (on remote service VNet)

Here, the resource is "remote service VNet hosting the remote service." (As mentioned in your verbatim)

Hope this clarifies.

Please Accept an answer if correct.

Original posters help the community find answers faster by identifying the correct answer.

Omer Cohen 40 Reputation points

2024-11-06T21:53:09.5633333+00:00

Hey @KapilAnanth-MSFT , Thanks for the clarifications. I have learned the hard way that to connect an Application Gateway to a private endpoint I must create a private link configuration, connected to an existing frontend configuration, otherwise i encounter an error: "No supported sub-resources" (see image below).

What I still don't get is why the private link configuration only able to choose the public frontend configuration and not the private one? Both of the frontend configurations are associated with a listener and are referenced by a rule (see image below). Perhaps it has to do with the fact that the private endpoint and application gateway are on different regions?

Lastly, is there a Python SDK to implement the deployment of a private link configuration for an application gateway? I only found documentation for a CLI-based solution.

Many thanks, Omer.

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-11-07T05:53:28.28+00:00

@Omer Cohen , glad we could be assistance.

Region should not be an issue

I was able to select Private IP of App Gateway for the Private EndPoint

Can you please verify you created the PLS for Private Frontend IP Configuration, and not for the Public IP Configuration?

From within the Application Gateway?

I was able to find a Python SDK for Azure Front Door but not for App Gateway.

I shall check the same for App Gateway and let you know soon

Cheers,

Kapil

Omer Cohen 40 Reputation points

2024-11-08T21:38:10.8766667+00:00

Thank you @KapilAnanth-MSFT ,
Eventually I managed to create the private link configuration for the private frontend IP configuration, but I have yet to test the suggested workflow, since I'm still implementing it (via SDK):
Private-endpoint (on VNet 'A') -> Application Gateway (on VNet 'B' via PLS) -> internal IP on remote cloud.

I assume this traffic should flow as intended thanks to a VPN gateway (on VNet 'B') that is built to route traffic destined to aforementioned internal IP (belonging to the app gateway's backend member) to its destination on the remote cloud.

What are your thoughts on my conjecture?
I appreciate your guidance with the matter.

Ganesh Patapati 2,435 Reputation points Microsoft Vendor

2024-11-11T14:45:17.43+00:00

Hey Omer Cohen,

I hope you are doing well!

As KapilAnanth-MSFT suggested, I would request that you test it; if you face any issues, please let us know so that we can work on this further.

Regards,

Ganesh

Omer Cohen 40 Reputation points

2024-11-11T20:41:01.76+00:00

Hey @Ganesh Patapati , @KapilAnanth-MSFT
I'm afraid my initial assumption could be correct: Application gateway is unable to communicate with VPN gateway due to a mismatch between the OSI levels (7 and 3).
in response to curl 10.52.0.4 from VM (see diagram below) I'm getting 502 Bad Gateway. I would assume the traffic will be redirected to the VPN gateway otherwise.

Additional Information:

Private frontend ip configuration appGatewayPrivateFrontendIP is associated with an http listener, listening on port 80.

Backend pool appGatewayBackendPool targets remote ip: 10.150.0.8. Tried adding the public ip of the VPN gateway as well, without success.

A local network gateway represents a remote with IP: 35.245.132.91, address space: 10.150.0.0/20 and associated with a successfully established VPN connection (to said remote).

A routing rule is associated with the above listener and backend pool. The rule's backend setting has an HTTP backend protocol and backend port: 80.

My backend pool is unhealthy, resulting in 502 error. I assume the application gateway can't probe a service operating outside of layer 7 via an http request.

I would greatly appreciate your assistance with the matter,
Omer.

Ganesh Patapati 2,435 Reputation points Microsoft Vendor

2024-11-13T09:57:27.15+00:00

Hey Omer Cohen,

We appreciate your patience!

How are the consumer VPC Vnet and Producer surrogate Vnet connected?

I don't see a Vnet peering or any connection between the 2 Vnets in the diagram.

Generally, in a hub and spoke config, when using Private endpoint, a Vnet peering or a VPN connection is needed between th.e Vnets.

Refer: https://learn.microsoft.com/en-us/azure/architecture/networking/guide/private-link-hub-spoke-network

Refer: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/private-link-and-dns-integration-at-scale

And in a hub and spoke config, all the Vnets are linked to the same Private DNS zone.

Refer: https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns-integration#virtual-network-workloads-without-custom-dns-server

I Hopes this clarifies,

If you are still facing any further issues, please don't hesitate to reach out to us. We are happy to assist you.

Looking forward to your response and appreciate your time on this.

Regards,

Ganesh

Omer Cohen 40 Reputation points

2024-11-13T12:26:00.5066667+00:00

Hey @Ganesh Patapati ,

Through the private link connection (see diagram).

No need, private endpoint forwards traffic via private link connection to the private link resource: Application gateway.

It isn't required in this scenario. No need to establish a secure connection. Secondly, VNet peering exposes the Consumer VPC, which we aim to isolate.

The issue we need to solve here is the traffic flow between the application gateway and the VPN gateway. How can we create the flow suggested by @KapilAnanth-MSFT ?
Private-endpoint (on VNet 'A') -> Application Gateway (on VNet 'B' via PLS) -> internal IP of remote resource -> Redirected to VPN (on VNet 'B').

Many thanks,
Omer.

Ganesh Patapati 2,435 Reputation points Microsoft Vendor

2024-11-15T15:18:10+00:00

Hey Omer Cohen

We appreciate your patience!

I sincerely suggest that you do the following tasks and let me know what the behavior is.

Create a VM or webapp within the same Vnet as Application gateway and add it behind the App gateway and then try accessing it from the consumer Vnet VM to check if that works.

Try to access the Application gateway IP or FQDN directly from the consumer Vnet VM and check if that works.

can you try to get clarity about where exactly it is not working.

If it is an issue with the remote connection or an issue with the App GW.

Regards,

Ganesh

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-11-19T04:56:26.2066667+00:00

Hi Omer Cohen,

Greetings.

I see you have tried to repo the same with a Load Balancer : https://learn.microsoft.com/en-us/answers/questions/2120456/troubleshooting-bad-requests-through-a-private-end

Wrt App gateway and your question

"The issue we need to solve here is the traffic flow between the application gateway and the VPN gateway. How can we create the flow suggested"

You should not worry about the routing

If you add the IP(of the resource not VPN Gateway) as backend, routing should be taken care by platform.

Can you address the following queries,

From a VM in same VNET as App Gateway, can you access the resource using App Gateway's private IP

From a VM in same VNET as App Gateway, can you access the resource using the resource's private IP (direct access)

And confirm if this is working or not?

Post this, we can troubleshoot for PLS.

I also see you mentioned, "My backend pool is unhealthy, resulting in 502 error. I assume the application gateway can't probe a service operating outside of layer 7 via an http request."

Isn't your backend pool (resource in VNETB) a HTTP/HTTPS Application?

Does this work on TCP/UDP?

Cheers,

Kapil

Cheers,

Kapil

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-11-20T03:07:24.3733333+00:00

Hi Omer Cohen,

For Azure Python SDK for App gateway, you can refer : https://learn.microsoft.com/en-in/python/api/azure-mgmt-network/azure.mgmt.network.networkmanagementclient?view=azure-python

Cheers,

Kapil

Omer Cohen 40 Reputation points

2024-11-20T15:03:05.1733333+00:00

Hey @KapilAnanth-MSFT ,

I'm now encountering an error when trying to add a private link configuration to an application gateway, either via CLI, or via web portal. Said error doesn't contain much info: (ApplicationGatewayPrivateLinkProvisioningError) An error occurred while configuring private link on Application Gateway..
The private link configuration rows are added after the error, but endpoints don't recognize it (probably due to the erroneous deployment), hence no sub-resource is found, resulting in being unable to be connected to a private-endpoint.
When Trying to alter/save the private link configuration I'm receiving an error: Failed to save configuration changes to application gateway 'app-gw-sky-ps-vpc-japaneast'. Error: An error occurred while configuring private link on Application Gateway.
Additional information for debugging:
The application gateway is created in a dedicated subnet.
The private link service has a dedicated subnet with private_link_service_network_policies: Disabled.
http listener is configured for a private frontend ip config of app gw.
app gw rule is asociated with the aforementioned listener, a backend pool and backend settings.

The provided link in your last message doesn't lead to Application Gateway SDK, although I'm familiar with it. I have yet to find an SDK alternative to az network application-gateway private-link add.

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-11-22T04:18:11.7533333+00:00

Hi Omer Cohen,

In that case, to troubleshoot further, we will need a specialized 1:1 session, where a support engineer can check the backend logs to pinpoint the issue.

If you have a support plan you may file a support ticket, else please check my latest private message on steps to avail an one time free support.

Wrt Python SDK, can you please check the below 2,

ApplicationGatewayPrivateLinkResourcesOperations Class

ApplicationGatewayPrivateEndpointConnectionsOperations Class

Cheers,

Kapil

Omer Cohen 40 Reputation points

2024-11-26T22:51:19.8433333+00:00

Hey @KapilAnanth-MSFT ,
I've deployed the resources shown in the diagram below.
This time I deployed the Application gateway manually, managing to avoid the error messages that appeared on Azure-Portal when I tried to save the private link configurations, previously created via Python SDK.

The code returns an unhelpful error code: ApplicationGatewayPrivateLinkProvisioningError, while it works perfectly via CLI.
Moreover, the private-link configurations are still added, but aren't detectable as sub-resources via the private-endpoint.

command = [ "az", "network", "application-gateway", "private-link", "add", "-g", RESOURCE_GROUP_NAME, "--gateway-name", app_gateway_name, "--name", private_link_config_name, "--frontend-ip", private_frontend_config_ip_name, "--subnet", subnet_name, "--primary" ] result = subprocess.run( command, capture_output=True, text=True, check=True)

The subnet provided above is created with the following flag: private_link_service_network_policies=Disabled, as required.

All in all, this is encouraging, but I wonder whether Azure support could help in this scenario (once I am assigned to the appropriate role).
Thanks,
Omer.

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-11-27T05:11:50.0533333+00:00

Hi Omer Cohen,

Azure Support has access to backend logs of your subscription.

So they can check why "ApplicationGatewayPrivateLinkProvisioningError" was thrown

Cheers,

Kapil

Omer Cohen 40 Reputation points

2024-12-07T16:13:22.88+00:00

The issue was the length of the name provided for the private link configuration.

Thank you for your time @KapilAnanth-MSFT .

KapilAnanth-MSFT 47,996 Reputation points Microsoft Employee

2024-12-08T14:28:02.99+00:00

Hi Omer Cohen,

Glad the issue has been resolved.

Thanks for your continued contribution on Q&A and appreciate much for taking the time to share your feedback

Cheers,

Kapil
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Exploring Cost-Effective Solutions for Routing Traffic from an Azure Private Endpoint to a VPN Gateway

0 additional answers

Your answer