Azure Application Gateway
An Azure service that provides a platform-managed, scalable, and highly available application delivery controller as a service.
1,099 questions
Backend Server Health Status Conflicting with Health Probe Logs in Application Gateway
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 43%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAE%3C/text%3E%3C/svg%3E)
Arunkumar Elango
0
Reputation points
Hello Azure Community,
I am encountering an issue with my Azure Application Gateway where the backend server status is reported as Healthy, but the health probe log indicates that the server cannot be reached. The log states:
"Cannot connect to the backend server. Check whether any NSG/UDR/Firewall is blocking access to the server. Check if the application is running on the correct port.
However, the instance health is displayed as Healthy in the pipeline results.
Here are some additional details:
- A Virtual Machine Scale Set (VMSS) is being used as the backend server.
- The health probe is configured correctly to target the expected endpoint.
- Network Security Groups (NSGs) and User-Defined Routes (UDRs) appear to be correctly configured, but the error persists.
Please provide insights on why this discrepancy might occur and steps to troubleshoot or resolve it.
Attached is the DevOps Pipeline log in image format for your reference. Please find it attached.
Thank you in advance...!
Regards,
Arunkumar Elango
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 71%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGP%3C/text%3E%3C/svg%3E)
Ganesh Patapati 2,515 Reputation points Microsoft Vendor2024-12-24T20:38:25.1233333+00:00 Greetings!
Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.
Message: Application Gateway could not connect to the backend. Check that the backend responds on the port used for the probe. Also check whether any NSG/UDR/Firewall is blocking access to the Ip and port of this backend.
Cause: After the DNS resolution phase, Application Gateway tries to connect to the backend server on the TCP port configured in the HTTP settings. If Application Gateway can't establish a TCP session on the port specified, the probe is marked as Unhealthy with this message.
Solution: If you receive this error, follow these steps:
- Check whether you can connect to the backend server on the port mentioned in the HTTP settings by using a browser or PowerShell. For example, run the following command:
Test-NetConnection -ComputerName www.bing.com -Port 443. - If the port mentioned isn't the desired port, enter the correct port number for Application Gateway to connect to the backend server.
- If you can't connect on the port from your local machine as well, then: a. Check the network security group (NSG) settings of the backend server's network adapter and subnet and whether inbound connections to the configured port are allowed. If they aren't, create a new rule to allow the connections. To learn how to create NSG rules, see the documentation page. b. Check whether the NSG settings of the Application Gateway subnet allow outbound public and private traffic, so that a connection can be made. Check the document page that's provided in step 3a to learn more about how to create NSG rules. Azure PowerShellCopy
c. Check the user-defined routes (UDR) settings of Application Gateway and the backend server's subnet for any routing anomalies. Make sure the UDR isn't directing the traffic away from the backend subnet. For example, check for routes to network virtual appliances or default routes being advertised to the Application Gateway subnet via Azure ExpressRoute and/or VPN. d. To check the effective routes and rules for a network adapter, you can use the following PowerShell commands: Azure PowerShellCopy$vnet = Get-AzVirtualNetwork -Name "vnetName" -ResourceGroupName "rgName" Get-AzVirtualNetworkSubnetConfig -Name appGwSubnet -VirtualNetwork $vnetGet-AzEffectiveNetworkSecurityGroup -NetworkInterfaceName "nic1" -ResourceGroupName "testrg" Get-AzEffectiveRouteTable -NetworkInterfaceName "nic1" -ResourceGroupName "testrg" - If you don't find any issues with NSG or UDR, check your backend server for application-related issues that are preventing clients from establishing a TCP session on the ports configured. A few things to check: a. Open a command prompt (Win+R -> cmd), enter netstat, and select Enter. b. Check whether the server is listening on the configured port. For example: Copy
c. If it's not listening on the configured port, check your web server settings. For example: site bindings in IIS, server block in NGINX and virtual host in Apache. d. Check your OS firewall settings to make sure that incoming traffic to the port is allowed.Proto Local Address Foreign Address State PID TCP 0.0.0.0:80 0.0.0.0:0 LISTENING 4
Additionally,
Here's what you can try:
- First validate whether the Application and VM are fine by trying to access the application from another VM in the same subnet.
- Next, try to get a VM in a different subnet and try to access the application, to mimic the AGW trying to connect to the backend. This will help you validate whether your NSGs are properly configured.
If above is unclear and/or you are unsure about something add a comment below.
Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.
Regards,
Ganesh
- Check whether you can connect to the backend server on the port mentioned in the HTTP settings by using a browser or PowerShell. For example, run the following command:
Sign in to comment
Sign in to answer