High CPU in monitoringhost.exe on Azure Virtual Machines (be careful what you wish for)

A customer recently told us as he was experiencing high CPU in two of Azure virtual machines that were operating as web servers. The process that was consuming the CPU was monitoringhost.exe, which is a child process of the Microsoft Monitoring Host service, our OpsMgr and Log Analytics Agent.

The job of this process is to do all the monitoring and data collection asked of it by the configuration of OpsMgr or Azure Log Analytics. I looked at what the process was up to and could see that it was busy running the following function

Microsoft_EnterpriseManagement_Mom_Modules_CloudFileUpload!Microsoft.EnterpriseManagement.Mom.Modules.CloudFileUpload.AsyncStreamHashCalculator.ReadStreamCallback

So, we were busy uploading a file to the workspace and calculating a hash on them in doing this.

Checking the process further. I could see we were uploading the following file.

C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Monitoring Host Temporary Files 319\45\W3SVC1-u_extend1.log.iislog

We were uploading an IIS log as you can easily ask Log Analytics to do for you.

We checked these IIS log files on the affected servers and found that they were 14 and 15 GB in size!

They had been configured to never rollover as in the following screenshot.

Changing these to rollover on a schedule managed the size of the log files and returned CPU usage to normal levels.

So, the moral of the story is to be careful when you enable additional data to be sent to OMS/Log Analytics. Take some time to verify what you are asking the service to do, and what data is due to be uploaded.

Brian McDermott
Senior Escalation Engineer
Microsoft

Comments

• Anonymous
  March 18, 2017
  What did you use to see the monitoringhost.exe ReadStreamCallback function was using CPU and that the file it was uploading was W3SVC1-u_extend1.log.iislog?
• Anonymous
  March 18, 2017
  And how did you get access to the server OS gui layer?