Microsoft.Insights dataCollectionRules 2022-06-01

Article
08/14/2024

Bicep resource definition

The dataCollectionRules resource type can be deployed with operations that target:

Resource groups - See resource group deployment commands

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.Insights/dataCollectionRules resource, add the following Bicep to your template.

resource symbolicname 'Microsoft.Insights/dataCollectionRules@2022-06-01' = {
  name: 'string'
  location: 'string'
  tags: {
    tagName1: 'tagValue1'
    tagName2: 'tagValue2'
  }
  kind: 'string'
  identity: {
    type: 'string'
    userAssignedIdentities: {
      {customized property}: {}
    }
  }
  properties: {
    dataCollectionEndpointId: 'string'
    dataFlows: [
      {
        builtInTransform: 'string'
        destinations: [
          'string'
        ]
        outputStream: 'string'
        streams: [
          'string'
        ]
        transformKql: 'string'
      }
    ]
    dataSources: {
      dataImports: {
        eventHub: {
          consumerGroup: 'string'
          name: 'string'
          stream: 'string'
        }
      }
      extensions: [
        {
          extensionName: 'string'
          extensionSettings: any()
          inputDataSources: [
            'string'
          ]
          name: 'string'
          streams: [
            'string'
          ]
        }
      ]
      iisLogs: [
        {
          logDirectories: [
            'string'
          ]
          name: 'string'
          streams: [
            'string'
          ]
        }
      ]
      logFiles: [
        {
          filePatterns: [
            'string'
          ]
          format: 'text'
          name: 'string'
          settings: {
            text: {
              recordStartTimestampFormat: 'string'
            }
          }
          streams: [
            'string'
          ]
        }
      ]
      performanceCounters: [
        {
          counterSpecifiers: [
            'string'
          ]
          name: 'string'
          samplingFrequencyInSeconds: int
          streams: [
            'string'
          ]
        }
      ]
      platformTelemetry: [
        {
          name: 'string'
          streams: [
            'string'
          ]
        }
      ]
      prometheusForwarder: [
        {
          labelIncludeFilter: {
            {customized property}: 'string'
          }
          name: 'string'
          streams: 'Microsoft-PrometheusMetrics'
        }
      ]
      syslog: [
        {
          facilityNames: [
            'string'
          ]
          logLevels: [
            'string'
          ]
          name: 'string'
          streams: 'Microsoft-Syslog'
        }
      ]
      windowsEventLogs: [
        {
          name: 'string'
          streams: [
            'string'
          ]
          xPathQueries: [
            'string'
          ]
        }
      ]
      windowsFirewallLogs: [
        {
          name: 'string'
          streams: [
            'string'
          ]
        }
      ]
    }
    description: 'string'
    destinations: {
      azureMonitorMetrics: {
        name: 'string'
      }
      eventHubs: [
        {
          eventHubResourceId: 'string'
          name: 'string'
        }
      ]
      eventHubsDirect: [
        {
          eventHubResourceId: 'string'
          name: 'string'
        }
      ]
      logAnalytics: [
        {
          name: 'string'
          workspaceResourceId: 'string'
        }
      ]
      monitoringAccounts: [
        {
          accountResourceId: 'string'
          name: 'string'
        }
      ]
      storageAccounts: [
        {
          containerName: 'string'
          name: 'string'
          storageAccountResourceId: 'string'
        }
      ]
      storageBlobsDirect: [
        {
          containerName: 'string'
          name: 'string'
          storageAccountResourceId: 'string'
        }
      ]
      storageTablesDirect: [
        {
          name: 'string'
          storageAccountResourceId: 'string'
          tableName: 'string'
        }
      ]
    }
    streamDeclarations: {
      {customized property}: {
        columns: [
          {
            name: 'string'
            type: 'string'
          }
        ]
      }
    }
  }
}

Property values

dataCollectionRules

Name	Description	Value
name	The resource name	string (required)
location	The geo-location where the resource lives.	string (required)
tags	Resource tags.	Dictionary of tag names and values. See Tags in templates
kind	The kind of the resource.	'Linux' 'Windows'
identity	Managed service identity of the resource.	DataCollectionRuleResourceIdentity
properties	Resource properties.	DataCollectionRuleResourceProperties

DataCollectionRuleResourceIdentity

Name	Description	Value
type	Type of managed service identity (where both SystemAssigned and UserAssigned types are allowed).	'None' 'SystemAssigned' 'SystemAssigned,UserAssigned' 'UserAssigned' (required)
userAssignedIdentities	The set of user assigned identities associated with the resource. The userAssignedIdentities dictionary keys will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}. The dictionary values can be empty objects ({}) in requests.	UserAssignedIdentities

UserAssignedIdentities

Name	Description	Value
{customized property}		UserAssignedIdentity

UserAssignedIdentity

This object doesn't contain any properties to set during deployment. All properties are ReadOnly.

DataCollectionRuleResourceProperties

Name	Description	Value
dataCollectionEndpointId	The resource ID of the data collection endpoint that this rule can be used with.	string
dataFlows	The specification of data flows.	DataFlow[]
dataSources	The specification of data sources. This property is optional and can be omitted if the rule is meant to be used via direct calls to the provisioned endpoint.	DataCollectionRuleDataSources
description	Description of the data collection rule.	string
destinations	The specification of destinations.	DataCollectionRuleDestinations
streamDeclarations	Declaration of custom streams used in this rule.	DataCollectionRuleStreamDeclarations

DataFlow

Name	Description	Value
builtInTransform	The builtIn transform to transform stream data	string
destinations	List of destinations for this data flow.	string[]
outputStream	The output stream of the transform. Only required if the transform changes data to a different stream.	string
streams	List of streams for this data flow.	String array containing any of: 'Microsoft-Event' 'Microsoft-InsightsMetrics' 'Microsoft-Perf' 'Microsoft-Syslog' 'Microsoft-WindowsEvent'
transformKql	The KQL query to transform stream data.	string

DataCollectionRuleDataSources

Name	Description	Value
dataImports	Specifications of pull based data sources	DataSourcesSpecDataImports
extensions	The list of Azure VM extension data source configurations.	ExtensionDataSource[]
iisLogs	The list of IIS logs source configurations.	IisLogsDataSource[]
logFiles	The list of Log files source configurations.	LogFilesDataSource[]
performanceCounters	The list of performance counter data source configurations.	PerfCounterDataSource[]
platformTelemetry	The list of platform telemetry configurations	PlatformTelemetryDataSource[]
prometheusForwarder	The list of Prometheus forwarder data source configurations.	PrometheusForwarderDataSource[]
syslog	The list of Syslog data source configurations.	SyslogDataSource[]
windowsEventLogs	The list of Windows Event Log data source configurations.	WindowsEventLogDataSource[]
windowsFirewallLogs	The list of Windows Firewall logs source configurations.	WindowsFirewallLogsDataSource[]

DataSourcesSpecDataImports

Name	Description	Value
eventHub	Definition of Event Hub configuration.	DataImportSourcesEventHub

DataImportSourcesEventHub

Name	Description	Value
consumerGroup	Event Hub consumer group name	string
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
stream	The stream to collect from EventHub	string

ExtensionDataSource

Name	Description	Value
extensionName	The name of the VM extension.	string (required)
extensionSettings	The extension settings. The format is specific for particular extension.	For Bicep, you can use the any() function.
inputDataSources	The list of data sources this extension needs data from.	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-Event' 'Microsoft-InsightsMetrics' 'Microsoft-Perf' 'Microsoft-Syslog' 'Microsoft-WindowsEvent'

IisLogsDataSource

Name	Description	Value
logDirectories	Absolute paths file location	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	IIS streams	string[] (required)

LogFilesDataSource

Name	Description	Value
filePatterns	File Patterns where the log files are located	string[] (required)
format	The data format of the log files	'text' (required)
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
settings	The log files specific settings.	LogFilesDataSourceSettings
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data source	string[] (required)

LogFilesDataSourceSettings

Name	Description	Value
text	Text settings	LogFileSettingsText

LogFileSettingsText

Name	Description	Value
recordStartTimestampFormat	One of the supported timestamp formats	'ISO 8601' 'M/D/YYYY HH:MM:SS AM/PM' 'MMM d hh:mm:ss' 'Mon DD, YYYY HH:MM:SS' 'YYYY-MM-DD HH:MM:SS' 'dd/MMM/yyyy:HH:mm:ss zzz' 'ddMMyy HH:mm:ss' 'yyMMdd HH:mm:ss' 'yyyy-MM-ddTHH:mm:ssK' (required)

PerfCounterDataSource

Name	Description	Value
counterSpecifiers	A list of specifier names of the performance counters you want to collect. Use a wildcard (*) to collect a counter for all instances. To get a list of performance counters on Windows, run the command 'typeperf'.	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
samplingFrequencyInSeconds	The number of seconds between consecutive counter measurements (samples).	int
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-InsightsMetrics' 'Microsoft-Perf'

PlatformTelemetryDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of platform telemetry streams to collect	string[] (required)

PrometheusForwarderDataSource

Name	Description	Value
labelIncludeFilter	The list of label inclusion filters in the form of label "name-value" pairs. Currently only one label is supported: 'microsoft_metrics_include_label'. Label values are matched case-insensitively.	PrometheusForwarderDataSourceLabelIncludeFilter
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to.	String array containing any of: 'Microsoft-PrometheusMetrics'

PrometheusForwarderDataSourceLabelIncludeFilter

Name	Description	Value
{customized property}		string

SyslogDataSource

Name	Description	Value
facilityNames	The list of facility names.	String array containing any of: '*' 'alert' 'audit' 'auth' 'authpriv' 'clock' 'cron' 'daemon' 'ftp' 'kern' 'local0' 'local1' 'local2' 'local3' 'local4' 'local5' 'local6' 'local7' 'lpr' 'mail' 'mark' 'news' 'nopri' 'ntp' 'syslog' 'user' 'uucp'
logLevels	The log levels to collect.	String array containing any of: '*' 'Alert' 'Critical' 'Debug' 'Emergency' 'Error' 'Info' 'Notice' 'Warning'
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-Syslog'

WindowsEventLogDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-Event' 'Microsoft-WindowsEvent'
xPathQueries	A list of Windows Event Log queries in XPATH format.	string[]

WindowsFirewallLogsDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	Firewall logs streams	string[] (required)

DataCollectionRuleDestinations

Name	Description	Value
azureMonitorMetrics	Azure Monitor Metrics destination.	DestinationsSpecAzureMonitorMetrics
eventHubs	List of Event Hubs destinations.	EventHubDestination[]
eventHubsDirect	List of Event Hubs Direct destinations.	EventHubDirectDestination[]
logAnalytics	List of Log Analytics destinations.	LogAnalyticsDestination[]
monitoringAccounts	List of monitoring account destinations.	MonitoringAccountDestination[]
storageAccounts	List of storage accounts destinations.	StorageBlobDestination[]
storageBlobsDirect	List of Storage Blob Direct destinations. To be used only for sending data directly to store from the agent.	StorageBlobDestination[]
storageTablesDirect	List of Storage Table Direct destinations.	StorageTableDestination[]

DestinationsSpecAzureMonitorMetrics

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

EventHubDestination

Name	Description	Value
eventHubResourceId	The resource ID of the event hub.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

EventHubDirectDestination

Name	Description	Value
eventHubResourceId	The resource ID of the event hub.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

LogAnalyticsDestination

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
workspaceResourceId	The resource ID of the Log Analytics workspace.	string

MonitoringAccountDestination

Name	Description	Value
accountResourceId	The resource ID of the monitoring account.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

StorageBlobDestination

Name	Description	Value
containerName	The container name of the Storage Blob.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
storageAccountResourceId	The resource ID of the storage account.	string

StorageTableDestination

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
storageAccountResourceId	The resource ID of the storage account.	string
tableName	The name of the Storage Table.	string

DataCollectionRuleStreamDeclarations

Name	Description	Value
{customized property}		StreamDeclaration

StreamDeclaration

Name	Description	Value
columns	List of columns used by data in this stream.	ColumnDefinition[]

ColumnDefinition

Name	Description	Value
name	The name of the column.	string
type	The type of the column data.	'boolean' 'datetime' 'dynamic' 'int' 'long' 'real' 'string'

Quickstart templates

The following quickstart templates deploy this resource type.

Template	Description
Deploy Darktrace Autoscaling vSensors	This template allows you to deploy an automatically autoscaling deployment of Darktrace vSensors
Data Collection Rule for Syslog	This template creates a data collection rule defining the data source (Syslog) and the destination workspace.

ARM template resource definition

The dataCollectionRules resource type can be deployed with operations that target:

Resource groups - See resource group deployment commands

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.Insights/dataCollectionRules resource, add the following JSON to your template.

{
  "type": "Microsoft.Insights/dataCollectionRules",
  "apiVersion": "2022-06-01",
  "name": "string",
  "location": "string",
  "tags": {
    "tagName1": "tagValue1",
    "tagName2": "tagValue2"
  },
  "kind": "string",
  "identity": {
    "type": "string",
    "userAssignedIdentities": {
      "{customized property}": {}
    }
  },
  "properties": {
    "dataCollectionEndpointId": "string",
    "dataFlows": [
      {
        "builtInTransform": "string",
        "destinations": [ "string" ],
        "outputStream": "string",
        "streams": [ "string" ],
        "transformKql": "string"
      }
    ],
    "dataSources": {
      "dataImports": {
        "eventHub": {
          "consumerGroup": "string",
          "name": "string",
          "stream": "string"
        }
      },
      "extensions": [
        {
          "extensionName": "string",
          "extensionSettings": {},
          "inputDataSources": [ "string" ],
          "name": "string",
          "streams": [ "string" ]
        }
      ],
      "iisLogs": [
        {
          "logDirectories": [ "string" ],
          "name": "string",
          "streams": [ "string" ]
        }
      ],
      "logFiles": [
        {
          "filePatterns": [ "string" ],
          "format": "text",
          "name": "string",
          "settings": {
            "text": {
              "recordStartTimestampFormat": "string"
            }
          },
          "streams": [ "string" ]
        }
      ],
      "performanceCounters": [
        {
          "counterSpecifiers": [ "string" ],
          "name": "string",
          "samplingFrequencyInSeconds": "int",
          "streams": [ "string" ]
        }
      ],
      "platformTelemetry": [
        {
          "name": "string",
          "streams": [ "string" ]
        }
      ],
      "prometheusForwarder": [
        {
          "labelIncludeFilter": {
            "{customized property}": "string"
          },
          "name": "string",
          "streams": "Microsoft-PrometheusMetrics"
        }
      ],
      "syslog": [
        {
          "facilityNames": [ "string" ],
          "logLevels": [ "string" ],
          "name": "string",
          "streams": "Microsoft-Syslog"
        }
      ],
      "windowsEventLogs": [
        {
          "name": "string",
          "streams": [ "string" ],
          "xPathQueries": [ "string" ]
        }
      ],
      "windowsFirewallLogs": [
        {
          "name": "string",
          "streams": [ "string" ]
        }
      ]
    },
    "description": "string",
    "destinations": {
      "azureMonitorMetrics": {
        "name": "string"
      },
      "eventHubs": [
        {
          "eventHubResourceId": "string",
          "name": "string"
        }
      ],
      "eventHubsDirect": [
        {
          "eventHubResourceId": "string",
          "name": "string"
        }
      ],
      "logAnalytics": [
        {
          "name": "string",
          "workspaceResourceId": "string"
        }
      ],
      "monitoringAccounts": [
        {
          "accountResourceId": "string",
          "name": "string"
        }
      ],
      "storageAccounts": [
        {
          "containerName": "string",
          "name": "string",
          "storageAccountResourceId": "string"
        }
      ],
      "storageBlobsDirect": [
        {
          "containerName": "string",
          "name": "string",
          "storageAccountResourceId": "string"
        }
      ],
      "storageTablesDirect": [
        {
          "name": "string",
          "storageAccountResourceId": "string",
          "tableName": "string"
        }
      ]
    },
    "streamDeclarations": {
      "{customized property}": {
        "columns": [
          {
            "name": "string",
            "type": "string"
          }
        ]
      }
    }
  }
}

Property values

dataCollectionRules

Name	Description	Value
type	The resource type	'Microsoft.Insights/dataCollectionRules'
apiVersion	The resource api version	'2022-06-01'
name	The resource name	string (required)
location	The geo-location where the resource lives.	string (required)
tags	Resource tags.	Dictionary of tag names and values. See Tags in templates
kind	The kind of the resource.	'Linux' 'Windows'
identity	Managed service identity of the resource.	DataCollectionRuleResourceIdentity
properties	Resource properties.	DataCollectionRuleResourceProperties

DataCollectionRuleResourceIdentity

Name	Description	Value
type	Type of managed service identity (where both SystemAssigned and UserAssigned types are allowed).	'None' 'SystemAssigned' 'SystemAssigned,UserAssigned' 'UserAssigned' (required)
userAssignedIdentities	The set of user assigned identities associated with the resource. The userAssignedIdentities dictionary keys will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}. The dictionary values can be empty objects ({}) in requests.	UserAssignedIdentities

UserAssignedIdentities

Name	Description	Value
{customized property}		UserAssignedIdentity

UserAssignedIdentity

This object doesn't contain any properties to set during deployment. All properties are ReadOnly.

DataCollectionRuleResourceProperties

Name	Description	Value
dataCollectionEndpointId	The resource ID of the data collection endpoint that this rule can be used with.	string
dataFlows	The specification of data flows.	DataFlow[]
dataSources	The specification of data sources. This property is optional and can be omitted if the rule is meant to be used via direct calls to the provisioned endpoint.	DataCollectionRuleDataSources
description	Description of the data collection rule.	string
destinations	The specification of destinations.	DataCollectionRuleDestinations
streamDeclarations	Declaration of custom streams used in this rule.	DataCollectionRuleStreamDeclarations

DataFlow

Name	Description	Value
builtInTransform	The builtIn transform to transform stream data	string
destinations	List of destinations for this data flow.	string[]
outputStream	The output stream of the transform. Only required if the transform changes data to a different stream.	string
streams	List of streams for this data flow.	String array containing any of: 'Microsoft-Event' 'Microsoft-InsightsMetrics' 'Microsoft-Perf' 'Microsoft-Syslog' 'Microsoft-WindowsEvent'
transformKql	The KQL query to transform stream data.	string

DataCollectionRuleDataSources

Name	Description	Value
dataImports	Specifications of pull based data sources	DataSourcesSpecDataImports
extensions	The list of Azure VM extension data source configurations.	ExtensionDataSource[]
iisLogs	The list of IIS logs source configurations.	IisLogsDataSource[]
logFiles	The list of Log files source configurations.	LogFilesDataSource[]
performanceCounters	The list of performance counter data source configurations.	PerfCounterDataSource[]
platformTelemetry	The list of platform telemetry configurations	PlatformTelemetryDataSource[]
prometheusForwarder	The list of Prometheus forwarder data source configurations.	PrometheusForwarderDataSource[]
syslog	The list of Syslog data source configurations.	SyslogDataSource[]
windowsEventLogs	The list of Windows Event Log data source configurations.	WindowsEventLogDataSource[]
windowsFirewallLogs	The list of Windows Firewall logs source configurations.	WindowsFirewallLogsDataSource[]

DataSourcesSpecDataImports

Name	Description	Value
eventHub	Definition of Event Hub configuration.	DataImportSourcesEventHub

DataImportSourcesEventHub

Name	Description	Value
consumerGroup	Event Hub consumer group name	string
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
stream	The stream to collect from EventHub	string

ExtensionDataSource

Name	Description	Value
extensionName	The name of the VM extension.	string (required)
extensionSettings	The extension settings. The format is specific for particular extension.
inputDataSources	The list of data sources this extension needs data from.	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-Event' 'Microsoft-InsightsMetrics' 'Microsoft-Perf' 'Microsoft-Syslog' 'Microsoft-WindowsEvent'

IisLogsDataSource

Name	Description	Value
logDirectories	Absolute paths file location	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	IIS streams	string[] (required)

LogFilesDataSource

Name	Description	Value
filePatterns	File Patterns where the log files are located	string[] (required)
format	The data format of the log files	'text' (required)
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
settings	The log files specific settings.	LogFilesDataSourceSettings
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data source	string[] (required)

LogFilesDataSourceSettings

Name	Description	Value
text	Text settings	LogFileSettingsText

LogFileSettingsText

Name	Description	Value
recordStartTimestampFormat	One of the supported timestamp formats	'ISO 8601' 'M/D/YYYY HH:MM:SS AM/PM' 'MMM d hh:mm:ss' 'Mon DD, YYYY HH:MM:SS' 'YYYY-MM-DD HH:MM:SS' 'dd/MMM/yyyy:HH:mm:ss zzz' 'ddMMyy HH:mm:ss' 'yyMMdd HH:mm:ss' 'yyyy-MM-ddTHH:mm:ssK' (required)

PerfCounterDataSource

Name	Description	Value
counterSpecifiers	A list of specifier names of the performance counters you want to collect. Use a wildcard (*) to collect a counter for all instances. To get a list of performance counters on Windows, run the command 'typeperf'.	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
samplingFrequencyInSeconds	The number of seconds between consecutive counter measurements (samples).	int
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-InsightsMetrics' 'Microsoft-Perf'

PlatformTelemetryDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of platform telemetry streams to collect	string[] (required)

PrometheusForwarderDataSource

Name	Description	Value
labelIncludeFilter	The list of label inclusion filters in the form of label "name-value" pairs. Currently only one label is supported: 'microsoft_metrics_include_label'. Label values are matched case-insensitively.	PrometheusForwarderDataSourceLabelIncludeFilter
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to.	String array containing any of: 'Microsoft-PrometheusMetrics'

PrometheusForwarderDataSourceLabelIncludeFilter

Name	Description	Value
{customized property}		string

SyslogDataSource

Name	Description	Value
facilityNames	The list of facility names.	String array containing any of: '*' 'alert' 'audit' 'auth' 'authpriv' 'clock' 'cron' 'daemon' 'ftp' 'kern' 'local0' 'local1' 'local2' 'local3' 'local4' 'local5' 'local6' 'local7' 'lpr' 'mail' 'mark' 'news' 'nopri' 'ntp' 'syslog' 'user' 'uucp'
logLevels	The log levels to collect.	String array containing any of: '*' 'Alert' 'Critical' 'Debug' 'Emergency' 'Error' 'Info' 'Notice' 'Warning'
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-Syslog'

WindowsEventLogDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: 'Microsoft-Event' 'Microsoft-WindowsEvent'
xPathQueries	A list of Windows Event Log queries in XPATH format.	string[]

WindowsFirewallLogsDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	Firewall logs streams	string[] (required)

DataCollectionRuleDestinations

Name	Description	Value
azureMonitorMetrics	Azure Monitor Metrics destination.	DestinationsSpecAzureMonitorMetrics
eventHubs	List of Event Hubs destinations.	EventHubDestination[]
eventHubsDirect	List of Event Hubs Direct destinations.	EventHubDirectDestination[]
logAnalytics	List of Log Analytics destinations.	LogAnalyticsDestination[]
monitoringAccounts	List of monitoring account destinations.	MonitoringAccountDestination[]
storageAccounts	List of storage accounts destinations.	StorageBlobDestination[]
storageBlobsDirect	List of Storage Blob Direct destinations. To be used only for sending data directly to store from the agent.	StorageBlobDestination[]
storageTablesDirect	List of Storage Table Direct destinations.	StorageTableDestination[]

DestinationsSpecAzureMonitorMetrics

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

EventHubDestination

Name	Description	Value
eventHubResourceId	The resource ID of the event hub.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

EventHubDirectDestination

Name	Description	Value
eventHubResourceId	The resource ID of the event hub.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

LogAnalyticsDestination

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
workspaceResourceId	The resource ID of the Log Analytics workspace.	string

MonitoringAccountDestination

Name	Description	Value
accountResourceId	The resource ID of the monitoring account.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

StorageBlobDestination

Name	Description	Value
containerName	The container name of the Storage Blob.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
storageAccountResourceId	The resource ID of the storage account.	string

StorageTableDestination

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
storageAccountResourceId	The resource ID of the storage account.	string
tableName	The name of the Storage Table.	string

DataCollectionRuleStreamDeclarations

Name	Description	Value
{customized property}		StreamDeclaration

StreamDeclaration

Name	Description	Value
columns	List of columns used by data in this stream.	ColumnDefinition[]

ColumnDefinition

Name	Description	Value
name	The name of the column.	string
type	The type of the column data.	'boolean' 'datetime' 'dynamic' 'int' 'long' 'real' 'string'

Quickstart templates

The following quickstart templates deploy this resource type.

Template	Description
Deploy Darktrace Autoscaling vSensors	This template allows you to deploy an automatically autoscaling deployment of Darktrace vSensors
Data Collection Rule for Syslog	This template creates a data collection rule defining the data source (Syslog) and the destination workspace.

Terraform (AzAPI provider) resource definition

The dataCollectionRules resource type can be deployed with operations that target:

Resource groups

For a list of changed properties in each API version, see change log.

Resource format

To create a Microsoft.Insights/dataCollectionRules resource, add the following Terraform to your template.

resource "azapi_resource" "symbolicname" {
  type = "Microsoft.Insights/dataCollectionRules@2022-06-01"
  name = "string"
  location = "string"
  parent_id = "string"
  tags = {
    tagName1 = "tagValue1"
    tagName2 = "tagValue2"
  }
  identity {
    type = "string"
    identity_ids = []
  }
  body = jsonencode({
    properties = {
      dataCollectionEndpointId = "string"
      dataFlows = [
        {
          builtInTransform = "string"
          destinations = [
            "string"
          ]
          outputStream = "string"
          streams = [
            "string"
          ]
          transformKql = "string"
        }
      ]
      dataSources = {
        dataImports = {
          eventHub = {
            consumerGroup = "string"
            name = "string"
            stream = "string"
          }
        }
        extensions = [
          {
            extensionName = "string"
            inputDataSources = [
              "string"
            ]
            name = "string"
            streams = [
              "string"
            ]
          }
        ]
        iisLogs = [
          {
            logDirectories = [
              "string"
            ]
            name = "string"
            streams = [
              "string"
            ]
          }
        ]
        logFiles = [
          {
            filePatterns = [
              "string"
            ]
            format = "text"
            name = "string"
            settings = {
              text = {
                recordStartTimestampFormat = "string"
              }
            }
            streams = [
              "string"
            ]
          }
        ]
        performanceCounters = [
          {
            counterSpecifiers = [
              "string"
            ]
            name = "string"
            samplingFrequencyInSeconds = int
            streams = [
              "string"
            ]
          }
        ]
        platformTelemetry = [
          {
            name = "string"
            streams = [
              "string"
            ]
          }
        ]
        prometheusForwarder = [
          {
            labelIncludeFilter = {
              {customized property} = "string"
            }
            name = "string"
            streams = "Microsoft-PrometheusMetrics"
          }
        ]
        syslog = [
          {
            facilityNames = [
              "string"
            ]
            logLevels = [
              "string"
            ]
            name = "string"
            streams = "Microsoft-Syslog"
          }
        ]
        windowsEventLogs = [
          {
            name = "string"
            streams = [
              "string"
            ]
            xPathQueries = [
              "string"
            ]
          }
        ]
        windowsFirewallLogs = [
          {
            name = "string"
            streams = [
              "string"
            ]
          }
        ]
      }
      description = "string"
      destinations = {
        azureMonitorMetrics = {
          name = "string"
        }
        eventHubs = [
          {
            eventHubResourceId = "string"
            name = "string"
          }
        ]
        eventHubsDirect = [
          {
            eventHubResourceId = "string"
            name = "string"
          }
        ]
        logAnalytics = [
          {
            name = "string"
            workspaceResourceId = "string"
          }
        ]
        monitoringAccounts = [
          {
            accountResourceId = "string"
            name = "string"
          }
        ]
        storageAccounts = [
          {
            containerName = "string"
            name = "string"
            storageAccountResourceId = "string"
          }
        ]
        storageBlobsDirect = [
          {
            containerName = "string"
            name = "string"
            storageAccountResourceId = "string"
          }
        ]
        storageTablesDirect = [
          {
            name = "string"
            storageAccountResourceId = "string"
            tableName = "string"
          }
        ]
      }
      streamDeclarations = {
        {customized property} = {
          columns = [
            {
              name = "string"
              type = "string"
            }
          ]
        }
      }
    }
    kind = "string"
  })
}

Property values

dataCollectionRules

Name	Description	Value
type	The resource type	"Microsoft.Insights/dataCollectionRules@2022-06-01"
name	The resource name	string (required)
location	The geo-location where the resource lives.	string (required)
parent_id	To deploy to a resource group, use the ID of that resource group.	string (required)
tags	Resource tags.	Dictionary of tag names and values.
kind	The kind of the resource.	"Linux" "Windows"
identity	Managed service identity of the resource.	DataCollectionRuleResourceIdentity
properties	Resource properties.	DataCollectionRuleResourceProperties

DataCollectionRuleResourceIdentity

Name	Description	Value
type	Type of managed service identity (where both SystemAssigned and UserAssigned types are allowed).	"SystemAssigned" "SystemAssigned,UserAssigned" "UserAssigned" (required)
identity_ids	The set of user assigned identities associated with the resource. The userAssignedIdentities dictionary keys will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}. The dictionary values can be empty objects ({}) in requests.	Array of user identity IDs.

UserAssignedIdentities

Name	Description	Value
{customized property}		UserAssignedIdentity

UserAssignedIdentity

This object doesn't contain any properties to set during deployment. All properties are ReadOnly.

DataCollectionRuleResourceProperties

Name	Description	Value
dataCollectionEndpointId	The resource ID of the data collection endpoint that this rule can be used with.	string
dataFlows	The specification of data flows.	DataFlow[]
dataSources	The specification of data sources. This property is optional and can be omitted if the rule is meant to be used via direct calls to the provisioned endpoint.	DataCollectionRuleDataSources
description	Description of the data collection rule.	string
destinations	The specification of destinations.	DataCollectionRuleDestinations
streamDeclarations	Declaration of custom streams used in this rule.	DataCollectionRuleStreamDeclarations

DataFlow

Name	Description	Value
builtInTransform	The builtIn transform to transform stream data	string
destinations	List of destinations for this data flow.	string[]
outputStream	The output stream of the transform. Only required if the transform changes data to a different stream.	string
streams	List of streams for this data flow.	String array containing any of: "Microsoft-Event" "Microsoft-InsightsMetrics" "Microsoft-Perf" "Microsoft-Syslog" "Microsoft-WindowsEvent"
transformKql	The KQL query to transform stream data.	string

DataCollectionRuleDataSources

Name	Description	Value
dataImports	Specifications of pull based data sources	DataSourcesSpecDataImports
extensions	The list of Azure VM extension data source configurations.	ExtensionDataSource[]
iisLogs	The list of IIS logs source configurations.	IisLogsDataSource[]
logFiles	The list of Log files source configurations.	LogFilesDataSource[]
performanceCounters	The list of performance counter data source configurations.	PerfCounterDataSource[]
platformTelemetry	The list of platform telemetry configurations	PlatformTelemetryDataSource[]
prometheusForwarder	The list of Prometheus forwarder data source configurations.	PrometheusForwarderDataSource[]
syslog	The list of Syslog data source configurations.	SyslogDataSource[]
windowsEventLogs	The list of Windows Event Log data source configurations.	WindowsEventLogDataSource[]
windowsFirewallLogs	The list of Windows Firewall logs source configurations.	WindowsFirewallLogsDataSource[]

DataSourcesSpecDataImports

Name	Description	Value
eventHub	Definition of Event Hub configuration.	DataImportSourcesEventHub

DataImportSourcesEventHub

Name	Description	Value
consumerGroup	Event Hub consumer group name	string
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
stream	The stream to collect from EventHub	string

ExtensionDataSource

Name	Description	Value
extensionName	The name of the VM extension.	string (required)
extensionSettings	The extension settings. The format is specific for particular extension.
inputDataSources	The list of data sources this extension needs data from.	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: "Microsoft-Event" "Microsoft-InsightsMetrics" "Microsoft-Perf" "Microsoft-Syslog" "Microsoft-WindowsEvent"

IisLogsDataSource

Name	Description	Value
logDirectories	Absolute paths file location	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	IIS streams	string[] (required)

LogFilesDataSource

Name	Description	Value
filePatterns	File Patterns where the log files are located	string[] (required)
format	The data format of the log files	"text" (required)
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
settings	The log files specific settings.	LogFilesDataSourceSettings
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data source	string[] (required)

LogFilesDataSourceSettings

Name	Description	Value
text	Text settings	LogFileSettingsText

LogFileSettingsText

Name	Description	Value
recordStartTimestampFormat	One of the supported timestamp formats	"ISO 8601" "M/D/YYYY HH:MM:SS AM/PM" "MMM d hh:mm:ss" "Mon DD, YYYY HH:MM:SS" "YYYY-MM-DD HH:MM:SS" "dd/MMM/yyyy:HH:mm:ss zzz" "ddMMyy HH:mm:ss" "yyMMdd HH:mm:ss" "yyyy-MM-ddTHH:mm:ssK" (required)

PerfCounterDataSource

Name	Description	Value
counterSpecifiers	A list of specifier names of the performance counters you want to collect. Use a wildcard (*) to collect a counter for all instances. To get a list of performance counters on Windows, run the command 'typeperf'.	string[]
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
samplingFrequencyInSeconds	The number of seconds between consecutive counter measurements (samples).	int
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: "Microsoft-InsightsMetrics" "Microsoft-Perf"

PlatformTelemetryDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of platform telemetry streams to collect	string[] (required)

PrometheusForwarderDataSource

Name	Description	Value
labelIncludeFilter	The list of label inclusion filters in the form of label "name-value" pairs. Currently only one label is supported: 'microsoft_metrics_include_label'. Label values are matched case-insensitively.	PrometheusForwarderDataSourceLabelIncludeFilter
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to.	String array containing any of: "Microsoft-PrometheusMetrics"

PrometheusForwarderDataSourceLabelIncludeFilter

Name	Description	Value
{customized property}		string

SyslogDataSource

Name	Description	Value
facilityNames	The list of facility names.	String array containing any of: "*" "alert" "audit" "auth" "authpriv" "clock" "cron" "daemon" "ftp" "kern" "local0" "local1" "local2" "local3" "local4" "local5" "local6" "local7" "lpr" "mail" "mark" "news" "nopri" "ntp" "syslog" "user" "uucp"
logLevels	The log levels to collect.	String array containing any of: "*" "Alert" "Critical" "Debug" "Emergency" "Error" "Info" "Notice" "Warning"
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: "Microsoft-Syslog"

WindowsEventLogDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	List of streams that this data source will be sent to. A stream indicates what schema will be used for this data and usually what table in Log Analytics the data will be sent to.	String array containing any of: "Microsoft-Event" "Microsoft-WindowsEvent"
xPathQueries	A list of Windows Event Log queries in XPATH format.	string[]

WindowsFirewallLogsDataSource

Name	Description	Value
name	A friendly name for the data source. This name should be unique across all data sources (regardless of type) within the data collection rule.	string
streams	Firewall logs streams	string[] (required)

DataCollectionRuleDestinations

Name	Description	Value
azureMonitorMetrics	Azure Monitor Metrics destination.	DestinationsSpecAzureMonitorMetrics
eventHubs	List of Event Hubs destinations.	EventHubDestination[]
eventHubsDirect	List of Event Hubs Direct destinations.	EventHubDirectDestination[]
logAnalytics	List of Log Analytics destinations.	LogAnalyticsDestination[]
monitoringAccounts	List of monitoring account destinations.	MonitoringAccountDestination[]
storageAccounts	List of storage accounts destinations.	StorageBlobDestination[]
storageBlobsDirect	List of Storage Blob Direct destinations. To be used only for sending data directly to store from the agent.	StorageBlobDestination[]
storageTablesDirect	List of Storage Table Direct destinations.	StorageTableDestination[]

DestinationsSpecAzureMonitorMetrics

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

EventHubDestination

Name	Description	Value
eventHubResourceId	The resource ID of the event hub.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

EventHubDirectDestination

Name	Description	Value
eventHubResourceId	The resource ID of the event hub.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

LogAnalyticsDestination

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
workspaceResourceId	The resource ID of the Log Analytics workspace.	string

MonitoringAccountDestination

Name	Description	Value
accountResourceId	The resource ID of the monitoring account.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string

StorageBlobDestination

Name	Description	Value
containerName	The container name of the Storage Blob.	string
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
storageAccountResourceId	The resource ID of the storage account.	string

StorageTableDestination

Name	Description	Value
name	A friendly name for the destination. This name should be unique across all destinations (regardless of type) within the data collection rule.	string
storageAccountResourceId	The resource ID of the storage account.	string
tableName	The name of the Storage Table.	string

DataCollectionRuleStreamDeclarations

Name	Description	Value
{customized property}		StreamDeclaration

StreamDeclaration

Name	Description	Value
columns	List of columns used by data in this stream.	ColumnDefinition[]

ColumnDefinition

Name	Description	Value
name	The name of the column.	string
type	The type of the column data.	"boolean" "datetime" "dynamic" "int" "long" "real" "string"

Share via

Microsoft.Insights dataCollectionRules 2022-06-01

Bicep resource definition

Resource format

Property values

dataCollectionRules

DataCollectionRuleResourceIdentity

UserAssignedIdentities

UserAssignedIdentity

DataCollectionRuleResourceProperties

DataFlow

DataCollectionRuleDataSources

DataSourcesSpecDataImports

DataImportSourcesEventHub

ExtensionDataSource

IisLogsDataSource

LogFilesDataSource

LogFilesDataSourceSettings

LogFileSettingsText

PerfCounterDataSource

PlatformTelemetryDataSource

PrometheusForwarderDataSource

PrometheusForwarderDataSourceLabelIncludeFilter

SyslogDataSource

WindowsEventLogDataSource

WindowsFirewallLogsDataSource

DataCollectionRuleDestinations

DestinationsSpecAzureMonitorMetrics

EventHubDestination

EventHubDirectDestination

LogAnalyticsDestination

MonitoringAccountDestination

StorageBlobDestination

StorageTableDestination

DataCollectionRuleStreamDeclarations

StreamDeclaration

ColumnDefinition

Quickstart templates

ARM template resource definition

Resource format

Property values

dataCollectionRules

DataCollectionRuleResourceIdentity

UserAssignedIdentities

UserAssignedIdentity

DataCollectionRuleResourceProperties

DataFlow

DataCollectionRuleDataSources

DataSourcesSpecDataImports

DataImportSourcesEventHub

ExtensionDataSource

IisLogsDataSource

LogFilesDataSource

LogFilesDataSourceSettings

LogFileSettingsText

PerfCounterDataSource

PlatformTelemetryDataSource

PrometheusForwarderDataSource

PrometheusForwarderDataSourceLabelIncludeFilter

SyslogDataSource

WindowsEventLogDataSource

WindowsFirewallLogsDataSource

DataCollectionRuleDestinations

DestinationsSpecAzureMonitorMetrics

EventHubDestination

EventHubDirectDestination

LogAnalyticsDestination

MonitoringAccountDestination

StorageBlobDestination

StorageTableDestination

DataCollectionRuleStreamDeclarations

StreamDeclaration

ColumnDefinition

Quickstart templates

Terraform (AzAPI provider) resource definition

Resource format

Property values

dataCollectionRules

DataCollectionRuleResourceIdentity

UserAssignedIdentities