Azure Synapse Analytics monitoring data reference
This article contains all the monitoring reference information for this service.
See Monitor Azure Synapse Analytics for details on the data you can collect for Azure Synapse Analytics and how to use it.
Metrics
This section lists all the automatically collected platform metrics for this service. These metrics are also part of the global list of all platform metrics supported in Azure Monitor.
For information on metric retention, see Azure Monitor Metrics overview.
Supported metrics for Microsoft.Synapse/workspaces
The following table lists the metrics available for the Microsoft.Synapse/workspaces resource type.
- All columns might not be present in every table.
- Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.
Table headings
- Category - The metrics group or classification.
- Metric - The metric display name as it appears in the Azure portal.
- Name in REST API - The metric name as referred to in the REST API.
- Unit - Unit of measure.
- Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
- Dimensions - Dimensions available for the metric.
- Time Grains - Intervals at which the metric is sampled. For example,
PT1Mindicates that the metric is sampled every minute,PT30Mevery 30 minutes,PT1Hevery hour, and so on. - DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.
Category: Built-in SQL Pool
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Data processed (bytes) Amount of data processed by queries |
BuiltinSqlPoolDataProcessedBytes |
Bytes | Total (Sum) | <none> | PT1M | No |
| Login attempts Count of login attempts that succeded or failed |
BuiltinSqlPoolLoginAttempts |
Count | Average, Minimum, Maximum, Total (Sum) | Result |
PT1M | No |
| Requests ended Count of Requests that succeeded, failed, or were cancelled |
BuiltinSqlPoolRequestsEnded |
Count | Average, Minimum, Maximum, Total (Sum) | Result |
PT1M | No |
Category: Integration
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Activity runs ended Count of integration activities that succeeded, failed, or were cancelled |
IntegrationActivityRunsEnded |
Count | Count, Total (Sum) | Result, FailureType, Activity, ActivityType, Pipeline |
PT1M, PT1H | No |
| Link connection events Number of Synapse Link connection events including start, stop and failure. |
IntegrationLinkConnectionEvents |
Count | Total (Sum) | EventType, LinkConnectionName |
PT1M | No |
| Link processed rows Changed row count processed by Synapse Link. |
IntegrationLinkProcessedChangedRows |
Count | Total (Sum) | TableName, LinkConnectionName |
PT1M | No |
| Link processed data volume (bytes) Data volume in bytes processed by Synapse Link. |
IntegrationLinkProcessedDataVolume |
Bytes | Total (Sum) | TableName, LinkTableStatus, LinkConnectionName |
PT1M | No |
| Link latency in seconds Synapse Link data processing latency in seconds. |
IntegrationLinkProcessingLatencyInSeconds |
Count | Maximum, Minimum, Average | LinkConnectionName |
PT1M | No |
| Link table events Number of Synapse Link table events including snapshot, removal and failure. |
IntegrationLinkTableEvents |
Count | Total (Sum) | TableName, EventType, LinkConnectionName |
PT1M | No |
| Pipeline runs ended Count of integration pipeline runs that succeeded, failed, or were cancelled |
IntegrationPipelineRunsEnded |
Count | Count, Total (Sum) | Result, FailureType, Pipeline |
PT1M, PT1H | No |
| Trigger Runs ended Count of integration triggers that succeeded, failed, or were cancelled |
IntegrationTriggerRunsEnded |
Count | Count, Total (Sum) | Result, FailureType, Trigger |
PT1M, PT1H | No |
Category: Streaming job
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Backlogged input events (preview) This is a preview metric available in East US, West Europe. Number of input events sources backlogged. |
SQLStreamingBackloggedInputEventSources |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Data conversion errors (preview) This is a preview metric available in East US, West Europe. Number of output events that could not be converted to the expected output schema. Error policy can be changed to 'Drop' to drop events that encounter this scenario. |
SQLStreamingConversionErrors |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Input deserialization errors (preview) This is a preview metric available in East US, West Europe. Number of input events that could not be deserialized. |
SQLStreamingDeserializationError |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Early input events (preview) This is a preview metric available in East US, West Europe. Number of input events which application time is considered early compared to arrival time, according to early arrival policy. |
SQLStreamingEarlyInputEvents |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Input event bytes (preview) This is a preview metric available in East US, West Europe. Amount of data received by the streaming job, in bytes. This can be used to validate that events are being sent to the input source. |
SQLStreamingInputEventBytes |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Input events (preview) This is a preview metric available in East US, West Europe. Number of input events. |
SQLStreamingInputEvents |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Input sources received (preview) This is a preview metric available in East US, West Europe. Number of input events sources per second. |
SQLStreamingInputEventsSourcesPerSecond |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Late input events (preview) This is a preview metric available in East US, West Europe. Number of input events which application time is considered late compared to arrival time, according to late arrival policy. |
SQLStreamingLateInputEvents |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Out of order events (preview) This is a preview metric available in East US, West Europe. Number of Event Hub Events (serialized messages) received by the Event Hub Input Adapter, received out of order that were either dropped or given an adjusted timestamp, based on the Event Ordering Policy. |
SQLStreamingOutOfOrderEvents |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Output events (preview) This is a preview metric available in East US, West Europe. Number of output events. |
SQLStreamingOutputEvents |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Watermark delay (preview) This is a preview metric available in East US, West Europe. Output watermark delay in seconds. |
SQLStreamingOutputWatermarkDelaySeconds |
Count | Maximum, Minimum, Average | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Resource % utilization (preview) This is a preview metric available in East US, West Europe. Resource utilization expressed as a percentage. High utilization indicates that the job is using close to the maximum allocated resources. |
SQLStreamingResourceUtilization |
Percent | Maximum, Minimum, Average | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
| Runtime errors (preview) This is a preview metric available in East US, West Europe. Total number of errors related to query processing (excluding errors found while ingesting events or outputting results). |
SQLStreamingRuntimeErrors |
Count | Total (Sum) | SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance |
PT1M | No |
Azure Synapse Link metrics
Azure Synapse Link emits the following metrics to Azure Monitor:
| Metric | Aggregation types | Description |
|---|---|---|
| Link connection events | Sum | Number of Synapse Link connection events, including start, stop, and failure |
| Link latency in seconds | Max, Min, Avg | Synapse Link data processing latency in seconds |
| Link processed data volume (bytes) | Sum | Data volume in bytes processed by Synapse Link |
| Link processed rows | Sum | Row counts processed by Synapse Link |
| Link table events | Sum | Number of Synapse Link table events, including snapshot, removal, and failure |
Supported metrics for Microsoft.Synapse/workspaces/bigDataPools
The following table lists the metrics available for the Microsoft.Synapse/workspaces/bigDataPools resource type.
- All columns might not be present in every table.
- Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.
Table headings
- Category - The metrics group or classification.
- Metric - The metric display name as it appears in the Azure portal.
- Name in REST API - The metric name as referred to in the REST API.
- Unit - Unit of measure.
- Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
- Dimensions - Dimensions available for the metric.
- Time Grains - Intervals at which the metric is sampled. For example,
PT1Mindicates that the metric is sampled every minute,PT30Mevery 30 minutes,PT1Hevery hour, and so on. - DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.
Category: Apache Spark pool
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| vCores allocated Allocated vCores for an Apache Spark Pool |
BigDataPoolAllocatedCores |
Count | Maximum, Minimum, Average, Total (Sum) | SubmitterId |
PT1M | No |
| Memory allocated (GB) Allocated Memory for Apach Spark Pool (GB) |
BigDataPoolAllocatedMemory |
Count | Maximum, Minimum, Average, Total (Sum) | SubmitterId |
PT1M | No |
| Active Apache Spark applications Total Active Apache Spark Pool Applications |
BigDataPoolApplicationsActive |
Count | Maximum, Minimum, Average | JobState |
PT1M | No |
| Ended Apache Spark applications Count of Apache Spark pool applications ended |
BigDataPoolApplicationsEnded |
Count | Total (Sum) | JobType, JobResult |
PT1M | No |
Supported metrics for Microsoft.Synapse/workspaces/kustoPools
The following table lists the metrics available for the Microsoft.Synapse/workspaces/kustoPools resource type.
- All columns might not be present in every table.
- Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.
Table headings
- Category - The metrics group or classification.
- Metric - The metric display name as it appears in the Azure portal.
- Name in REST API - The metric name as referred to in the REST API.
- Unit - Unit of measure.
- Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
- Dimensions - Dimensions available for the metric.
- Time Grains - Intervals at which the metric is sampled. For example,
PT1Mindicates that the metric is sampled every minute,PT30Mevery 30 minutes,PT1Hevery hour, and so on. - DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.
Category: Cluster health
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Cache utilization (deprecated) Utilization level in the cluster scope. The metric is deprecated and presented for backward compatibility only, you should use the 'Cache utilization factor' metric instead. |
CacheUtilization |
Percent | Average, Maximum, Minimum | <none> | PT1M | Yes |
| Cache utilization factor Percentage of utilized disk space dedicated for hot cache in the cluster. 100% means that the disk space assigned to hot data is optimally utilized. No action is needed in terms of the cache size. More than 100% means that the cluster's disk space is not large enough to accommodate the hot data, as defined by your caching policies. To ensure that sufficient space is available for all the hot data, the amount of hot data needs to be reduced or the cluster needs to be scaled out. Enabling auto scale is recommended. |
CacheUtilizationFactor |
Percent | Average, Maximum, Minimum | <none> | PT1M | Yes |
| CPU CPU utilization level |
CPU |
Percent | Average, Maximum, Minimum | <none> | PT1M | Yes |
| FollowerLatency The follower databases synchronize changes in the leader databases. Because of the synchronization, there's a data lag of a few seconds to a few minutes in data availability.This metric measures the length of the time lag. The time lag depends on the overall size of the leader database metadata.This is a cluster level metrics: the followers catch metadata of all databases that are followed. This metric represents the latency of the process. |
FollowerLatency |
MilliSeconds | Average, Maximum, Minimum | State, RoleInstance |
PT1M | Yes |
| Ingestion utilization Ratio of used ingestion slots in the cluster |
IngestionUtilization |
Percent | Average, Maximum, Minimum | <none> | PT1M | Yes |
| Instance Count Total instance count |
InstanceCount |
Count | Average, Maximum, Minimum, Count, Total (Sum) | <none> | PT1M | Yes |
| Keep alive Sanity check indicates the cluster responds to queries |
KeepAlive |
Count | Average | <none> | PT1M | Yes |
| Total number of extents Total number of data extents |
TotalNumberOfExtents |
Count | Average, Maximum, Minimum | <none> | PT1M | Yes |
| Total number of throttled commands Total number of throttled commands |
TotalNumberOfThrottledCommands |
Count | Average, Maximum, Minimum, Total (Sum) | CommandType |
PT1M | Yes |
Category: Export health and performance
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Continuous Export Max Lateness The lateness (in minutes) reported by the continuous export jobs in the cluster |
ContinuousExportMaxLatenessMinutes |
Count | Maximum | <none> | PT1M | Yes |
| Continuous export - num of exported records Number of records exported, fired for every storage artifact written during the export operation |
ContinuousExportNumOfRecordsExported |
Count | Total (Sum) | ContinuousExportName, Database |
PT1M | Yes |
| Continuous Export Pending Count The number of pending continuous export jobs ready for execution |
ContinuousExportPendingCount |
Count | Maximum | <none> | PT1M | Yes |
| Continuous Export Result Indicates whether Continuous Export succeeded or failed |
ContinuousExportResult |
Count | Count | ContinuousExportName, Result, Database |
PT1M | Yes |
| Export Utilization Export utilization |
ExportUtilization |
Percent | Maximum | <none> | PT1M | Yes |
Category: Ingestion health and performance
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Batch Blob Count Number of data sources in an aggregated batch for ingestion. |
BatchBlobCount |
Count | Average, Maximum, Minimum | Database |
PT1M | Yes |
| Batch Duration The duration of the aggregation phase in the ingestion flow. |
BatchDuration |
Seconds | Average, Maximum, Minimum | Database |
PT1M | Yes |
| Batches Processed Number of batches aggregated for ingestion. Batching Type: whether the batch reached batching time, data size or number of files limit set by batching policy |
BatchesProcessed |
Count | Total (Sum), Average, Maximum, Minimum | Database, SealReason |
PT1M | Yes |
| Batch Size Uncompressed expected data size in an aggregated batch for ingestion. |
BatchSize |
Bytes | Average, Maximum, Minimum | Database |
PT1M | Yes |
| Blobs Dropped Number of blobs permanently rejected by a component. |
BlobsDropped |
Count | Total (Sum), Average, Minimum, Maximum | Database, ComponentType, ComponentName |
PT1M | Yes |
| Blobs Processed Number of blobs processed by a component. |
BlobsProcessed |
Count | Total (Sum), Average, Minimum, Maximum | Database, ComponentType, ComponentName |
PT1M | Yes |
| Blobs Received Number of blobs received from input stream by a component. |
BlobsReceived |
Count | Total (Sum), Average, Minimum, Maximum | Database, ComponentType, ComponentName |
PT1M | Yes |
| Discovery Latency Reported by data connections (if exist). Time in seconds from when a message is enqueued or event is created until it is discovered by data connection. This time is not included in the Azure Data Explorer total ingestion duration. |
DiscoveryLatency |
Seconds | Average | ComponentType, ComponentName |
PT1M | Yes |
| Events Dropped Number of events dropped permanently by data connection. An Ingestion result metric with a failure reason will be sent. |
EventsDropped |
Count | Total (Sum), Average, Minimum, Maximum | ComponentType, ComponentName |
PT1M | Yes |
| Events Processed Number of events processed by the cluster |
EventsProcessed |
Count | Total (Sum), Average, Minimum, Maximum | ComponentType, ComponentName |
PT1M | Yes |
| Events Received Number of events received by data connection. |
EventsReceived |
Count | Total (Sum), Average, Minimum, Maximum | ComponentType, ComponentName |
PT1M | Yes |
| Ingestion Latency Latency of data ingested, from the time the data was received in the cluster until it's ready for query. The ingestion latency period depends on the ingestion scenario. |
IngestionLatencyInSeconds |
Seconds | Average, Maximum, Minimum | IngestionKind |
PT1M | Yes |
| Ingestion result Total number of sources that either failed or succeeded to be ingested. Splitting the metric by status, you can get detailed information about the status of the ingestion operations. |
IngestionResult |
Count | Total (Sum) | IngestionResultDetails, FailureKind |
PT1M | Yes |
| Ingestion Volume Overall volume of ingested data to the cluster |
IngestionVolumeInMB |
Bytes | Total (Sum), Maximum | Database |
PT1M | Yes |
| Queue Length Number of pending messages in a component's queue. |
QueueLength |
Count | Average | ComponentType |
PT1M | Yes |
| Queue Oldest Message Time in seconds from when the oldest message in queue was inserted. |
QueueOldestMessage |
Count | Average | ComponentType |
PT1M | Yes |
| Received Data Size Bytes Size of data received by data connection. This is the size of the data stream, or of raw data size if provided. |
ReceivedDataSizeBytes |
Bytes | Average, Total (Sum) | ComponentType, ComponentName |
PT1M | Yes |
| Stage Latency Cumulative time from when a message is discovered until it is received by the reporting component for processing (discovery time is set when message is enqueued for ingestion queue, or when discovered by data connection). |
StageLatency |
Seconds | Average | Database, ComponentType |
PT1M | Yes |
Category: Materialized View health and performance
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Materialized View Age The materialized view age in minutes |
MaterializedViewAgeMinutes |
Count | Average | Database, MaterializedViewName |
PT1M | Yes |
| Materialized View Age The materialized view age in seconds |
MaterializedViewAgeSeconds |
Seconds | Average, Minimum, Maximum | Database, MaterializedViewName |
PT1M | Yes |
| Materialized View Data Loss Indicates potential data loss in materialized view |
MaterializedViewDataLoss |
Count | Maximum | Database, MaterializedViewName, Kind |
PT1M | Yes |
| Materialized View Extents Rebuild Number of extents rebuild |
MaterializedViewExtentsRebuild |
Count | Average | Database, MaterializedViewName |
PT1M | Yes |
| Materialized View Health The health of the materialized view (1 for healthy, 0 for non-healthy) |
MaterializedViewHealth |
Count | Average | Database, MaterializedViewName |
PT1M | Yes |
| Materialized View Records In Delta The number of records in the non-materialized part of the view |
MaterializedViewRecordsInDelta |
Count | Average | Database, MaterializedViewName |
PT1M | Yes |
| Materialized View Result The result of the materialization process |
MaterializedViewResult |
Count | Average | Database, MaterializedViewName, Result |
PT1M | Yes |
Category: Partitioning
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Partitioning Percentage Percentage of records partitioned versus total number of records. |
PartitioningPercentage |
Percent | Average, Maximum, Minimum | Database, Table |
PT1M | Yes |
| Partitioning Percentage Hot Percentage of records partitioned versus total number of records (in hot / cached extents only). |
PartitioningPercentageHot |
Percent | Average, Maximum, Minimum | Database, Table |
PT1M | Yes |
| Processed Partitioned Records Number of records partitioned in measured time window. |
ProcessedPartitionedRecords |
Count | Average, Maximum, Minimum | Database, Table |
PT1M | Yes |
Category: Query performance
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Query duration Queries duration in seconds |
QueryDuration |
MilliSeconds | Average, Maximum, Minimum, Total (Sum) | QueryStatus |
PT1M | Yes |
| Query Result Total number of queries. |
QueryResult |
Count | Count | QueryStatus |
PT1M | No |
| Total number of concurrent queries Total number of concurrent queries |
TotalNumberOfConcurrentQueries |
Count | Average, Maximum, Minimum | <none> | PT1M | Yes |
| Total number of throttled queries Total number of throttled queries |
TotalNumberOfThrottledQueries |
Count | Average, Maximum, Minimum, Total (Sum) | <none> | PT1M | Yes |
| Weak consistency latency The max latency between the previous metadata sync and the next one (in DB/node scope) |
WeakConsistencyLatency |
Seconds | Average, Maximum, Minimum | Database, RoleInstance |
PT1M | Yes |
Category: Streaming Ingest
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Streaming Ingest Data Rate Streaming ingest data rate |
StreamingIngestDataRate |
Bytes | Average, Minimum, Maximum | <none> | PT1M | Yes |
| Streaming Ingest Duration Streaming ingest duration in milliseconds |
StreamingIngestDuration |
MilliSeconds | Average, Minimum, Maximum | <none> | PT1M | Yes |
| Streaming Ingest Result Streaming ingest result |
StreamingIngestResults |
Count | Count | Result |
PT1M | Yes |
| Streaming Ingest Utilization Streaming Ingest Utilization is the percentage of actual concurrent streaming ingestion requests performed, compared to the maximum number of concurrent streaming ingestion requests. |
StreamingIngestUtilization |
Percent | Average, Maximum, Minimum | <none> | PT1M | Yes |
Supported metrics for Microsoft.Synapse/workspaces/scopePools
The following table lists the metrics available for the Microsoft.Synapse/workspaces/scopePools resource type.
- All columns might not be present in every table.
- Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.
Table headings
- Category - The metrics group or classification.
- Metric - The metric display name as it appears in the Azure portal.
- Name in REST API - The metric name as referred to in the REST API.
- Unit - Unit of measure.
- Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
- Dimensions - Dimensions available for the metric.
- Time Grains - Intervals at which the metric is sampled. For example,
PT1Mindicates that the metric is sampled every minute,PT30Mevery 30 minutes,PT1Hevery hour, and so on. - DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.
Category: SCOPE pool
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| PN duration of SCOPE job PN (process node) duration (Milliseconds) used by each SCOPE job |
ScopePoolJobPNMetric |
Milliseconds | Maximum, Minimum, Average, Total (Sum), Count | JobType, JobResult |
PT1M | Yes |
| Queued duration of SCOPE job Queued duration (Milliseconds) used by each SCOPE job |
ScopePoolJobQueuedDurationMetric |
Milliseconds | Maximum, Minimum, Average, Total (Sum), Count | JobType |
PT1M | Yes |
| Running duration of SCOPE job Running duration (Milliseconds) used by each SCOPE job |
ScopePoolJobRunningDurationMetric |
Milliseconds | Maximum, Minimum, Average, Total (Sum), Count | JobType, JobResult |
PT1M | Yes |
Supported metrics for Microsoft.Synapse/workspaces/sqlPools
The following table lists the metrics available for the Microsoft.Synapse/workspaces/sqlPools resource type.
- All columns might not be present in every table.
- Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.
Table headings
- Category - The metrics group or classification.
- Metric - The metric display name as it appears in the Azure portal.
- Name in REST API - The metric name as referred to in the REST API.
- Unit - Unit of measure.
- Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
- Dimensions - Dimensions available for the metric.
- Time Grains - Intervals at which the metric is sampled. For example,
PT1Mindicates that the metric is sampled every minute,PT30Mevery 30 minutes,PT1Hevery hour, and so on. - DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.
Category: SQL dedicated pool
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Active queries The active queries. Using this metric unfiltered and unsplit displays all active queries running on the system |
ActiveQueries |
Count | Total (Sum) | IsUserDefined |
PT1M | No |
| Adaptive cache hit percentage Measures how well workloads are utilizing the adaptive cache. Use this metric with the cache hit percentage metric to determine whether to scale for additional capacity or rerun workloads to hydrate the cache |
AdaptiveCacheHitPercent |
Percent | Maximum, Minimum, Average | <none> | PT1M | No |
| Adaptive cache used percentage Measures how well workloads are utilizing the adaptive cache. Use this metric with the cache used percentage metric to determine whether to scale for additional capacity or rerun workloads to hydrate the cache |
AdaptiveCacheUsedPercent |
Percent | Maximum, Minimum, Average | <none> | PT1M | No |
| Connections Count of Total logins to the SQL pool |
Connections |
Count | Total (Sum) | Result |
PT1M | Yes |
| Connections blocked by firewall Count of connections blocked by firewall rules. Revisit access control policies for your SQL pool and monitor these connections if the count is high |
ConnectionsBlockedByFirewall |
Count | Total (Sum) | <none> | PT1M | No |
| CPU used percentage CPU utilization across all nodes in the SQL pool |
CPUPercent |
Percent | Maximum, Minimum, Average | <none> | PT1M | No |
| DWU limit Service level objective of the SQL pool |
DWULimit |
Count | Maximum, Minimum, Average | <none> | PT1M | No |
| DWU used Represents a high-level representation of usage across the SQL pool. Measured by DWU limit * DWU percentage |
DWUUsed |
Count | Maximum, Minimum, Average | <none> | PT1M | No |
| DWU used percentage Represents a high-level representation of usage across the SQL pool. Measured by taking the maximum between CPU percentage and Data IO percentage |
DWUUsedPercent |
Percent | Maximum, Minimum, Average | <none> | PT1M | No |
| Local tempdb used percentage Local tempdb utilization across all compute nodes - values are emitted every five minute |
LocalTempDBUsedPercent |
Percent | Maximum, Minimum, Average | <none> | PT1M | No |
| Memory used percentage Memory utilization across all nodes in the SQL pool |
MemoryUsedPercent |
Percent | Maximum, Minimum, Average | <none> | PT1M | No |
| Queued queries Cumulative count of requests queued after the max concurrency limit was reached |
QueuedQueries |
Count | Total (Sum) | IsUserDefined |
PT1M | No |
Category: SQL dedicated pool - Workload management
| Metric | Name in REST API | Unit | Aggregation | Dimensions | Time Grains | DS Export |
|---|---|---|---|---|---|---|
| Workload group active queries The active queries within the workload group. Using this metric unfiltered and unsplit displays all active queries running on the system |
WLGActiveQueries |
Count | Total (Sum) | IsUserDefined, WorkloadGroup |
PT1M | No |
| Workload group query timeouts Queries for the workload group that have timed out. Query timeouts reported by this metric are only once the query has started executing (it does not include wait time due to locking or resource waits) |
WLGActiveQueriesTimeouts |
Count | Total (Sum) | IsUserDefined, WorkloadGroup |
PT1M | No |
| Workload group allocation by max resource percent Displays the percentage allocation of resources relative to the Effective cap resource percent per workload group. This metric provides the effective utilization of the workload group |
WLGAllocationByEffectiveCapResourcePercent |
Percent | Maximum, Minimum, Average | IsUserDefined, WorkloadGroup |
PT1M | No |
| Workload group allocation by system percent The percentage allocation of resources relative to the entire system |
WLGAllocationBySystemPercent |
Percent | Maximum, Minimum, Average, Total (Sum) | IsUserDefined, WorkloadGroup |
PT1M | No |
| Effective cap resource percent The effective cap resource percent for the workload group. If there are other workload groups with min_percentage_resource > 0, the effective_cap_percentage_resource is lowered proportionally |
WLGEffectiveCapResourcePercent |
Percent | Maximum, Minimum, Average | IsUserDefined, WorkloadGroup |
PT1M | No |
| Effective min resource percent The effective min resource percentage setting allowed considering the service level and the workload group settings. The effective min_percentage_resource can be adjusted higher on lower service levels |
WLGEffectiveMinResourcePercent |
Percent | Minimum, Maximum, Average, Total (Sum) | IsUserDefined, WorkloadGroup |
PT1M | No |
| Workload group queued queries Cumulative count of requests queued after the max concurrency limit was reached |
WLGQueuedQueries |
Count | Total (Sum) | IsUserDefined, WorkloadGroup |
PT1M | No |
Details
Dedicated SQL pool measures performance in compute data warehouse units (DWUs). Rather than surfacing details of individual nodes such as memory per node or number of CPUs per node, metrics such as
MemoryUsedPercentandCPUPercentshow general usage trend over a period of time. These trends help administrators understand how a dedicated SQL pool instance is utilized. Changes in memory or CPU footprint could be a trigger for actions such as scale-up or scale-down of DWUs, or investigating queries that might require optimization.DWUUsedrepresents only high-level usage across the SQL pool and isn't a comprehensive indicator of utilization. To determine whether to scale up or down, consider all factors that DWU can impact, such as concurrency, memory, tempdb size, and adaptive cache capacity. Run your workload at different DWU settings to determine what works best to meet your business objectives.MemoryUsedPercentreflects utilization even if the data warehouse is idle, not active workload memory consumption. Track this metric along with tempdb size and Gen2 cache to decide whether you need to scale for more cache capacity to increase workload performance.Failed and successful connections are reported for a particular data warehouse, not for the server itself.
Metric dimensions
For information about what metric dimensions are, see Multi-dimensional metrics.
This service has the following dimensions associated with its metrics.
Microsoft.Synapse/workspaces
Result, FailureType, Activity, ActivityType, Pipeline, Trigger, EventType, TableName, LinkTableStatus, LinkConnectionName, SQLPoolName, SQLDatabaseName, JobName, LogicalName, PartitionId, ProcessorInstance
Use the Result dimension of the IntegrationActivityRunsEnded, IntegrationPipelineRunsEnded, IntegrationTriggerRunsEnded, and BuiltinSqlPoolDataRequestsEnded metrics to filter by Succeeded, Failed, or Canceled final state.
Microsoft.Synapse/workspaces/bigDataPools
SubmitterId, JobState, JobType, JobResult
Microsoft.Synapse/workspaces/kustoPools
Database, SealReason, ComponentType, ComponentName, ContinuousExportName, Result, EventStatus, State, RoleInstance, IngestionResultDetails, FailureKind, MaterializedViewName, Kind, Result, QueryStatus, ComponentType, CommandType
Microsoft.Synapse/workspaces/scopePools
JobType, JobResult
Microsoft.Synapse/workspaces/sqlPools
IsUserDefined, Result
Resource logs
This section lists the types of resource logs you can collect for this service. The section pulls from the list of all resource logs category types supported in Azure Monitor.
Supported resource logs for Microsoft.Synapse/workspaces
| Category | Category display name | Log table | Supports basic log plan | Supports ingestion-time transformation | Example queries | Costs to export |
|---|---|---|---|---|---|---|
BuiltinSqlReqsEnded |
Built-in Sql Pool Requests Ended | SynapseBuiltinSqlPoolRequestsEnded Ended Azure Synapse built-in serverless SQL requests. |
No | Yes | No | |
GatewayApiRequests |
Synapse Gateway Api Requests | SynapseGatewayApiRequests Azure Synapse gateway API requests. |
No | Yes | No | |
IntegrationActivityRuns |
Integration Activity Runs | SynapseIntegrationActivityRuns Logs for Synapse integration activity runs. |
No | Yes | Yes | |
IntegrationPipelineRuns |
Integration Pipeline Runs | SynapseIntegrationPipelineRuns Logs for Synapse integration pipeline runs. |
No | Yes | Yes | |
IntegrationTriggerRuns |
Integration Trigger Runs | SynapseIntegrationTriggerRuns Logs for Synapse integration trigger runs. |
No | Yes | Yes | |
SQLSecurityAuditEvents |
SQL Security Audit Event | SQLSecurityAuditEvents Azure Synapse SQL Audit Log. |
No | Yes | No | |
SynapseLinkEvent |
Synapse Link Event | SynapseLinkEvent Information about Synapse Link, including Link status and Link table status. |
No | No | Queries | Yes |
SynapseRbacOperations |
Synapse RBAC Operations | SynapseRbacOperations Azure Synapse role-based access control (SRBAC) operations. |
No | Yes | No |
Note
The event SynapseBuiltinSqlPoolRequestsEnded is emitted only for queries that read data from storage. It's not emitted for queries that process only metadata.
Supported resource logs for Microsoft.Synapse/workspaces/bigDataPools
| Category | Category display name | Log table | Supports basic log plan | Supports ingestion-time transformation | Example queries | Costs to export |
|---|---|---|---|---|---|---|
BigDataPoolAppEvents |
Big Data Pool Applications Execution Metrics | No | No | Yes | ||
BigDataPoolAppsEnded |
Big Data Pool Applications Ended | SynapseBigDataPoolApplicationsEnded Information about ended Apache Spark applications. |
No | Yes | No | |
BigDataPoolBlockManagerEvents |
Big Data Pool Block Manager Events | No | No | Yes | ||
BigDataPoolDriverLogs |
Big Data Pool Driver Logs | No | No | Yes | ||
BigDataPoolEnvironmentEvents |
Big Data Pool Environment Events | No | No | Yes | ||
BigDataPoolExecutorEvents |
Big Data Pool Executor Events | No | No | Yes | ||
BigDataPoolExecutorLogs |
Big Data Pool Executor Logs | No | No | Yes | ||
BigDataPoolJobEvents |
Big Data Pool Job Events | No | No | Yes | ||
BigDataPoolSqlExecutionEvents |
Big Data Pool Sql Execution Events | No | No | Yes | ||
BigDataPoolStageEvents |
Big Data Pool Stage Events | No | No | Yes | ||
BigDataPoolTaskEvents |
Big Data Pool Task Events | No | No | Yes |
Supported resource logs for Microsoft.Synapse/workspaces/kustoPools
| Category | Category display name | Log table | Supports basic log plan | Supports ingestion-time transformation | Example queries | Costs to export |
|---|---|---|---|---|---|---|
Command |
Command | SynapseDXCommand Azure data explorer synapse command execution summary. Logs include DatabaseName, State, Duration that can be used for monitoring the commands which were invoked on the cluster |
No | No | Yes | |
DataOperation |
Data operation | No | No | Yes | ||
FailedIngestion |
Failed ingestion | No | No | Yes | ||
IngestionBatching |
Ingestion batching | SynapseDXIngestionBatching Azure data explore synapse ingestion batching operations. These logs have detailed statistics of batches ready for ingestion (duration, batch size and blobs count) |
No | No | Yes | |
Journal |
Journal | No | No | Yes | ||
Query |
Query | SynapseDXQuery Azure data explorer synpase query execution summary. Logs include DatabaseName, State, Duration that can be used for monitoring the queries which were invoked on the cluster |
No | No | Yes | |
SucceededIngestion |
Succeeded ingestion | SynapseDXSucceededIngestion Succeeded ingestion operations logs provide information about successfully completed ingest operations. Logs include data source details that together with |
No | Yes | Yes | |
TableDetails |
Table details | SynapseDXTableDetails Azure Data Explorer Synpase table details |
No | No | Yes | |
TableUsageStatistics |
Table usage statistics | SynapseDXTableUsageStatistics Azure date explorer synapse table usage statistics. Logs include DatabaseName, TableName, User that can be used for monitoring cluster's table usage |
No | No | Yes |
Supported resource logs for Microsoft.Synapse/workspaces/scopePools
| Category | Category display name | Log table | Supports basic log plan | Supports ingestion-time transformation | Example queries | Costs to export |
|---|---|---|---|---|---|---|
ScopePoolScopeJobsEnded |
Scope Pool Scope Jobs Ended | SynapseScopePoolScopeJobsEnded SCOPE ended event including SCOPE job result and Information about the job. |
No | No | Yes | |
ScopePoolScopeJobsStateChange |
Scope Pool Scope Jobs State Change | No | No | Yes |
Supported resource logs for Microsoft.Synapse/workspaces/sqlPools
| Category | Category display name | Log table | Supports basic log plan | Supports ingestion-time transformation | Example queries | Costs to export |
|---|---|---|---|---|---|---|
DmsWorkers |
Dms Workers | SynapseSqlPoolDmsWorkers Information about workers completing DMS steps in an Azure Synapse dedicated SQL pool. |
Yes | Yes | No | |
ExecRequests |
Exec Requests | SynapseSqlPoolExecRequests Information about SQL requests or queries in an Azure Synapse dedicated SQL pool. |
Yes | Yes | No | |
RequestSteps |
Request Steps | SynapseSqlPoolRequestSteps Information about request steps that compose a given SQL request or query in an Azure Synapse dedicated SQL pool. |
Yes | Yes | No | |
SqlRequests |
Sql Requests | SynapseSqlPoolSqlRequests Information about query distributions of the steps of SQL requests/queries in an Azure Synapse dedicated SQL pool. |
Yes | Yes | No | |
SQLSecurityAuditEvents |
Sql Security Audit Event | SQLSecurityAuditEvents Azure Synapse SQL Audit Log. |
No | Yes | No | |
Waits |
Waits | SynapseSqlPoolWaits Information about the wait states encountered during execution of a SQL request/query in an Azure Synapse dedicated SQL pool, including locks and waits on transmission queues. |
Yes | Yes | No |
Dynamic Management Views (DMVs)
For more information on these logs, see the following information:
- sys.dm_pdw_exec_requests
- sys.dm_pdw_request_steps
- sys.dm_pdw_dms_workers
- sys.dm_pdw_waits
- sys.dm_pdw_sql_requests
To view the list of DMVs that apply to Synapse SQL, see System views supported in Synapse SQL.
Azure Monitor Logs tables
This section lists the Azure Monitor Logs tables relevant to this service, which are available for query by Log Analytics using Kusto queries. The tables contain resource log data and possibly more depending on what is collected and routed to them.
Synapse Workspaces
Microsoft.Synapse/workspaces
- AzureActivity
- SynapseRbacOperations
- SynapseGatewayApiRequests
- SynapseSqlPoolExecRequests
- SynapseSqlPoolRequestSteps
- SynapseSqlPoolDmsWorkers
- SynapseSqlPoolWaits
- SynapseSqlPoolSqlRequests
- SynapseIntegrationPipelineRuns
- SynapseLinkEvent
- SynapseIntegrationActivityRuns
- SynapseIntegrationTriggerRuns
- SynapseBigDataPoolApplicationsEnded
- SynapseBuiltinSqlPoolRequestsEnded
- SQLSecurityAuditEvents
- SynapseScopePoolScopeJobsEnded
- SynapseScopePoolScopeJobsStateChange
- SynapseDXCommand
- SynapseDXFailedIngestion
- SynapseDXIngestionBatching
- SynapseDXQuery
- SynapseDXSucceededIngestion
- SynapseDXTableUsageStatistics
- SynapseDXTableDetails
Available Apache Spark configurations
| Configuration name | Default value | Description |
|---|---|---|
| spark.synapse.logAnalytics.enabled | false | To enable the Log Analytics sink for the Spark applications, true. Otherwise, false. |
| spark.synapse.logAnalytics.workspaceId | - | The destination Log Analytics workspace ID. |
| spark.synapse.logAnalytics.secret | - | The destination Log Analytics workspace secret. |
| spark.synapse.logAnalytics.keyVault.linkedServiceName | - | The Key Vault linked service name for the Log Analytics workspace ID and key. |
| spark.synapse.logAnalytics.keyVault.name | - | The Key Vault name for the Log Analytics ID and key. |
| spark.synapse.logAnalytics.keyVault.key.workspaceId | SparkLogAnalyticsWorkspaceId | The Key Vault secret name for the Log Analytics workspace ID. |
| spark.synapse.logAnalytics.keyVault.key.secret | SparkLogAnalyticsSecret | The Key Vault secret name for the Log Analytics workspace |
| spark.synapse.logAnalytics.uriSuffix | ods.opinsights.azure.com | The destination Log Analytics workspace URI suffix. If your workspace isn't in Azure global, you need to update the URI suffix according to the respective cloud. |
| spark.synapse.logAnalytics.filter.eventName.match | - | Optional. The comma-separated spark event names, you can specify which events to collect. For example: SparkListenerJobStart,SparkListenerJobEnd |
| spark.synapse.logAnalytics.filter.loggerName.match | - | Optional. The comma-separated log4j logger names, you can specify which logs to collect. For example: org.apache.spark.SparkContext,org.example.Logger |
| spark.synapse.logAnalytics.filter.metricName.match | - | Optional. The comma-separated spark metric name suffixes, you can specify which metrics to collect. For example: jvm.heap.used |
Note
- For Microsoft Azure operated by 21Vianet, the
spark.synapse.logAnalytics.uriSuffixparameter should beods.opinsights.azure.cn. - For Azure Government, the
spark.synapse.logAnalytics.uriSuffixparameter should beods.opinsights.azure.us. - For any cloud except Azure, the
spark.synapse.logAnalytics.keyVault.nameparameter should be the fully qualified domain name (FQDN) of the Key Vault. For example,AZURE_KEY_VAULT_NAME.vault.usgovcloudapi.netfor AzureUSGovernment.
Activity log
The linked table lists the operations that can be recorded in the activity log for this service. These operations are a subset of all the possible resource provider operations in the activity log.
For more information on the schema of activity log entries, see Activity Log schema.
Related content
- See Monitor Azure Synapse Analytics for a description of monitoring Synapse Analytics.
- See Monitor Azure resources with Azure Monitor for details on monitoring Azure resources.