How can I determine where all these User Errors are coming from in Service Bus?

Question

Looking at metrics for the Service Bus, the amount of User Errors that are occurring do not add up to the errors that we are seeing for the processes that take place due to the service bus. Due to this we are concerned that there are a large number of errors occurring that we are just not aware of. For example, according to Service Bus metrics over a 24 hour period there are 14.29k User Errors categorized as "-NamespaceOnlyMetric-" and 7 other categories (one per each topic belonging to the Service Bus Namespace) totaling only 134 User Errors. How does the entire namespace calculate 14.29k User Errors vs the 134 for all 7 topics combined? I certainly don't see this many errors while looking at errors or failures in Application Insights or Log Analytics. Is there a query I need to run to view more detail on these User Errors? This is basically happening for every Service Bus instance in each environment for our Organization and it doesn't make sense to us. See picture for reference.

Answer 1

Hi Nieman, Josh
Welcome to Microsoft Q&A platform, Thanks for asking question here...!

To resolve the discrepancy between the User Errors in Azure Service Bus metrics and those in Application Insights or Log Analytics, it's important to understand how these metrics are calculated and what they represent.

Understanding User Errors in Azure Service Bus: User Errors in Azure Service Bus occur when requests are not processed due to client-side issues, such as invalid message formats or missing required properties. Examples include exceptions like MessageLockLostException and HeaderSizeExceeded.

Namespace vs. Topic Metrics: The 14.29k User Errors labeled as "-NamespaceOnlyMetric-" likely represent errors at the namespace level, affecting multiple topics rather than being tied to individual topics.

Granularity of Metrics: The 134 User Errors for individual topics may not capture all errors occurring at the namespace level, especially if they are related to configuration or operational limits affecting the entire namespace.

To gain more insight into these User Errors, consider running queries in Azure Monitor or Log Analytics that specifically filter for user error types.

You can use Kusto Query Language (KQL) within Azure Monitor or Log Analytics to explore user errors in detail. An example query might look like this:

AzureDiagnostics
| where ResourceType == "SERVICEBUSNAMESPACES"
| where OperationName == "UserErrors"
| summarize Count = count() by bin(TimeGenerated, 1h)
| order by TimeGenerated desc

You can also filter for specific exception types that may be contributing to the high user error count. For instance:

AzureDiagnostics
| where ResourceType == "SERVICEBUSNAMESPACES"
| where ExceptionType in ("MessageLockLostException", "HeaderSizeExceeded", "TTLExpiredException")
| summarize Count = count() by ExceptionType

For Reference: https://turbo360.com/blog/azure-service-bus-exception-handling-using-turbo360 https://particular.net/webinars/deep-dive-into-azure-service-bus-messaging

Please let us know, If you required anything.