Logs missing for a period of time from Application Insights

Question

In my current workspace I have 8 applications insights logging custom information for 8 different applications.

They have run for +2 years with no change to the logging code. Over the weekend for 3 application for a period of 12hours 2minutes there are no logs at all and then logs started to flow again.

Server was running, other 5 application did not present that issue, Application Insights and workspace are locate in East US 2. Last log was at 12/7/2024 3:43:28 AM UTC and the next log was at 12/7/2024 3:45:22 PM UTC.

During that period of time there are missing somewhat 500k logs as my applications keep running. As a note, for policy the logs are recorded in SQL and Application Insights as redundancy. In SQL the logs are there but missing in App Insights.

Has anyone experience any similar behavior? How did you fix it?

Thanks in advance!

Answer 1

Hi @David F. Mejia Cascante
Welcome to the Microsoft Q&A Platform! Thank you for asking your question here. 

It could be that there was a temporary issue with Azure usage in the US 2 eastern region, or maybe your applications were generating more logs than usual, hit a rate limit in Application Insights and could be a network issue preventing logging sending way temporarily, or there is an error in your logging setup fits the data it doesn't capture 

To check, I suggest checking Azure Service Health to see if there were any outages in your region during the affected period. You can also check Application Insights diagnostics for any errors or throttle symptoms that could have affected log ingestion. If you can’t find anything there, reviewing your network connection and logging configuration may also provide some answers. If the problem persists, I suggest you contact Azure Support to investigate the issue further. 

In the long term, you may want to consider setting up alerts in Application Insights to notify you if telemetry stops or if logs suddenly decrease, so you can quickly catch any issues. Additionally, if your applications generate a lot of logs, optimizing the transmitted data or spreading it across multiple workstations can help avoid hitting intake limits in the future. 

For reference, please review this documentation :-

https://learn.microsoft.com/en-us/troubleshoot/azure/azure-functions/monitoring/functions-monitoring-appinsightslogs

If you have any further queries, do let us know.

Answer 2

Hi David F. Mejia Cascante,

You are correct. Since some applications are able to send data from the same server, this confirms that it is not a Network/Firewall issue.

Regarding Daily Cap and Throttling, please refer to the following article:

https://learn.microsoft.com/en-us/troubleshoot/azure/azure-monitor/app-insights/availability/daily-cap-unexpected-behaviors#cause-1-application-insights-and-log-analytics-now-have-separate-daily-cap-implementations

Additionally, Microsoft provides guidance on missing or incorrect Application Insights logs for Azure Functions apps. You may find this helpful:

https://learn.microsoft.com/en-us/troubleshoot/azure/azure-functions/monitoring/functions-monitoring-appinsightslogs?source=recommendations

To further investigate what might be blocking logs or causing missing telemetry, and Look for any sudden spikes or high failure rate

Please check the following details in Application Insights:

1)Are there any HTTP requests in the Application Queue?

2)Are there any exceptions recorded during the failure time?

Check Application Insights Metrics

Go to Azure Portal → Application Insights → Metrics

Select "Requests", "Failed Requests", and "Server Response Time"

Feel free to reach out if you have any further questions or need additional information—I’m happy to assist!

Please provide your valuable comments

Please do not forget to "Accept the answer” and “upvote it” wherever the information provided helps you, this can be beneficial to other community members.it would be greatly appreciated and helpful to others.