Azure Monitor integration
Note
We will retire Azure HDInsight on AKS on January 31, 2025. Before January 31, 2025, you will need to migrate your workloads to Microsoft Fabric or an equivalent Azure product to avoid abrupt termination of your workloads. The remaining clusters on your subscription will be stopped and removed from the host.
Only basic support will be available until the retirement date.
Important
This feature is currently in preview. The Supplemental Terms of Use for Microsoft Azure Previews include more legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability. For information about this specific preview, see Azure HDInsight on AKS preview information. For questions or feature suggestions, please submit a request on AskHDInsight with the details and follow us for more updates on Azure HDInsight Community.
HDInsight on AKS offers an integration with Azure Monitor that can be used to monitor cluster pools and their clusters.
Azure Monitor collects metrics and logs from multiple resources into an Azure Monitor Log Analytics workspace, which presents the data as structured, queryable tables that can be used to configure custom alerts. Azure Monitor logs provide an excellent overall experience for monitoring workloads and interacting with logs, especially if you have multiple clusters.
Setting up Azure Monitor alerts is easy and beneficial. These alerts are triggered when the value of a metric or the results of a query meet certain conditions. A severity level for the alert can be added in addition to the name. The ability to specify severity is a powerful tool that can be used when creating multiple alerts. You can learn more about this topic and how to set up alerts here.
The integration offers various other capabilities such as a flexible canvas for data analysis and the creation of rich visual reports using Workbooks and as well cross cluster monitoring.
Azure HDInsight on AKS comes with integrated monitoring experience with Azure services like Azure managed Prometheus along with Azure managed Grafana dashboards for monitoring.
- Azure Managed Prometheus is a service that monitors your cloud environments. The monitoring is to maintain their availability, performance, and cluster metrics. It collects data generated by resources in your Azure instances and from other monitoring tools. The data is used to provide analysis across multiple sources.
- Azure Managed Grafana is a data visualization platform built on top of the Grafana software by Grafana Labs. It's built as a fully managed Azure service operated and supported by Microsoft. Grafana helps you bring together metrics, logs, and traces into a single user interface. With its extensive support for data sources and graphing capabilities, you can view and analyze your application and infrastructure telemetry data in real-time.
HDInsight on AKS also offers an out-of-box monitoring feature that provides premade dashboards based on cluster and service health information on top of the Azure monitor integration for more flexibility and a better visualization experience.
For more information