All logs from related to Spark including Jupyter and Livy logs.
Table attributes
Attribute
Value
Resource types
microsoft.hdinsight/clusters
Categories
Azure Resources
Solutions
LogManagement
Basic log
No
Ingestion-time transformation
Yes
Sample Queries
-
Columns
Column
Type
Description
_BilledSize
real
The record size in bytes
ClusterName
string
Name of cluster.
CorrelationId
string
The ID for correlated events. Can be used to identify correlated events between multiple tables.
HostName
string
Name of host where log was emitted.
_IsBillable
string
Specifies whether ingesting the data is billable. When _IsBillable is false ingestion isn't billed to your Azure account
LogLevel
string
log level of message (INFO, WARN, ERROR, etc.).
LogType
string
The name of the log file that a record came from (e.g. SparkExecutorLog, SparkDriverLog).
Message
string
message from HBase log.
OperationName
string
The operation associated with log record.
_ResourceId
string
A unique identifier for the resource that the record is associated with
SourceSystem
string
The type of agent the event was collected by. For example, OpsManager for Windows agent, either direct connect or Operations Manager, Linux for all Linux agents, or Azure for Azure Diagnostics
_SubscriptionId
string
A unique identifier for the subscription that the record is associated with
TenantId
string
The Log Analytics workspace ID
TimeGenerated
datetime
The timestamp (UTC) of when the log was generated.