Windows AzureMonitorAgent not sending heartbeat or any logs or any metrics.

Question

Many of our Windows Machines are not reporting their logs (to log analytics workspace) or metrics (to Azure monitor) after they were configured to use Azure Monitor agent for Windows. We can see that the process “amaexthealthmonitor.exe” running but it fails to call/initiate “MonAgentCore.exe” and hence none of the DCR’s have been processed and no heartbeat seen in the Log analytics workspace or in Azure Monitor Metrics. The VMs are using a User Managed Identity that is associated to the VM's and has got the rights in Log Analytics Workspace and Azure Monitor.

We have tried the following steps.

1. Removed the DCR association and then Removed the extension.
2. Renamed the following folders =>
   C:\WindowsAzure\Resources to C:\WindowsAzure\Resources.old
   C:\WindowsAzure\Logs\Plugins\Microsoft.Azure.Monitor.AzureMonitorWindowsAgent to C:\WindowsAzure\Logs\Plugins\Microsoft.Azure.Monitor.AzureMonitorWindowsAgent.old
3. Restarted the VM.
4. Added the Extension back.
   5., Associated the VM to the DCR

Anyone has any idea as to what the issue is here. The OS on these VM's are 2016 and 2019 (Most of them 2019).

Answer 1

Have seen this under multiple conditions:

• Firewall rules blocking "global.handler.control.monitor.azure.com:443" or "<region>.handler.control.monitor.azure.com:443"
• Agent provisioning state of the agent is either "Transitioning" or "Failed" or "Provision state is Successful but the "Handler Status" is Not Ready"
• VMInsights: Dependency Agent version MUST be 9.10 or above and configured as "Enabled for AMA"

The first and third are fairly straight forward to identify and resolve. The one that is unusual in many differenct scenarios is the second one as I've found in some cases it appears the Agent did not complete the provisioning configuration and had to do some one of the following in order:

1. Remove the VM from the DCR(s)
2. Remove the Azure Monitor Agent and wait 5 min.
3. Reinstall the azure monitor agent and check for heartbeat with: Set-AzVMextension - Name 'AzureMonitorWindowsAgent - ExtensionType 'AzureMonitorWindowsAgent' -Publisher Microsoft.Azure.Monitor -ResourceGroupName <rg name> -VMName <vm name> -Location < vm location> -TypeHandlerVersion "1.26" -EnableAutomaticUpgrade $true -SettingString '{"authentication":{managedIdentity":"identifier-name":"mi_res_id","identifier-value":"<UMI resource id>"}}}'
4. If get a heartbeat then re-associate the VM to the Data Collection Rule and monitor Logs ingestion
5. If NO heartbeat, open task manager and check if "moniAgentCore.exe" process is running and if it is then wait 5-10 more minutes for a heartbeat. If Not you have to check the local machine logs for errors.
6. If still no heartbeat, then remove the azure agent extension again and wait 5-10 min.
7. Manually associate the VIrtual machine to the "Data Collection Rule"
   1. The DCR will check for the existance of the Azure Monitor Agent and if not, it will install it for you. (Please note it will not configure the agent with the UMI)
   2. Wait 5-10 min. again and monitor for heartbeat and monitor for logs
8. If this doesn't resolve your issue you should contact MSFT support as their are other things that can be done to address this but should be provided to you from MSFT support.

I hope these step help as they are based upon the following article: https://learn.microsoft.com/en-us/azure/azure-monitor/agents/azure-monitor-agent-troubleshoot-windows-vm