Edit

Share via

Configure Microsoft Defender XDR to stream Advanced Hunting events to your Azure Event Hub

  • Article

Applies to:

Note

Try our new APIs using MS Graph security API. Find out more at: Use the Microsoft Graph security API - Microsoft Graph | Microsoft Learn.

Important

Some information in this article relates to a prereleased product which may be substantially modified before it's commercially released. Microsoft makes no warranties, expressed or implied, with respect to the information provided here.

Prerequisites

Prior to configuring Microsoft Defender XDR to stream data to Event Hubs, ensure the following prerequisites are fulfilled:

  1. Create an Event Hubs (for information, see Set up Event Hubs).

  2. Creating an Event Hubs Namespace (for information, see Set up Event Hubs namespace).

  3. Add permissions to the entity who has the privileges of a Contributor so that this entity can export data to the Event Hubs. For more information on adding permissions, see Add permissions

Note

The Streaming API can be integrated either via Event Hubs or Azure Storage Account.

Enable raw data streaming

  1. Log on to Microsoft Defender portal as a Security Administrator at a minimum.

Important

Microsoft recommends that you use roles with the fewest permissions. Using lower permissioned accounts helps improve security for your organization. Global Administrator is a highly privileged role that should be limited to emergency scenarios when you can't use an existing role.

  1. Go to the Streaming API settings page.

  2. Click on Add.

  3. Choose a name for your new settings.

  4. Choose Forward events to Azure Event Hub.

  5. You can select if you want to export the event data to a single Event Hub, or to export each event table to a different Event Hubs in your Event Hubs namespace.

  6. To export the event data to a single Event Hub, enter your Event Hub name and your Event Hub Namespace resource ID.

    To get your Event Hub Namespace resource ID, go to your Azure Event Hubs namespace page on Azure > Properties tab > copy the text under Resource ID:

    An Event Hub resource ID

  7. Go to the Supported Microsoft Defender XDR event types in event streaming API to review the support status of event types in the Microsoft 365 Streaming API.

  8. Choose the events you want to stream and click Save.

The schema of the events in Azure Event Hub

{
   "records": [
               {
                  "time": "<The time Microsoft Defender XDR received the event>"
                  "tenantId": "<The Id of the tenant that the event belongs to>"
                  "category": "<The Advanced Hunting table name with 'AdvancedHunting-' prefix>"
                  "properties": { <Microsoft Defender XDR Advanced Hunting event as Json> }
               }
               ...
            ]
}

  • Each Event Hubs message in Azure Event Hubs contains list of records.

  • Each record contains the event name, the time Microsoft Defender XDR received the event, the tenant it belongs (you'll only get events from your tenant), and the event in JSON format in a property called "properties".

  • For more information about the schema of Microsoft Defender XDR events, see Advanced Hunting overview.

  • In Advanced Hunting, the DeviceInfo table has a column named MachineGroup which contains the group of the device. Here every event will be decorated with this column as well.

Data types mapping

To get the data types for event properties, do the following steps:

  1. Log on to Microsoft Defender XDR and go to Advanced Hunting page.

  2. Run the following query to get the data types mapping for each event:

    {EventType}
| getschema
| project ColumnName, ColumnType

  • Here's an example for Device Info event:

    An example query for device info

Estimating initial Event Hub capacity

The following advanced hunting query can help provide a rough estimate of data volume throughput and initial event hub capacity based on events/sec and estimated MB/sec. We recommend running the query during regular business hours so as to capture 'real' throughput.

let bytes_ = 1000;
union withsource=MDTables MyDefenderTable // TODO: Insert desired tables one by one separated by a comma (for example: DeviceEvents, DeviceInfo) or with a wildcard (Device*)
| where Timestamp > startofday(ago(7d))
| summarize count() by bin(Timestamp, 1m), MDTables
| extend EPS = count_ /60 
| summarize avg(EPS), estimatedMBPerSec = avg(EPS) * bytes_ / (1024*1024) by MDTables, bin(Timestamp, 3h)
| summarize avg_EPS=max(avg_EPS), estimatedMBPerSec = max(estimatedMBPerSec) by MDTables
| sort by toint(estimatedMBPerSec) desc
| project MDTables, avg_EPS, estimatedMBPerSec

To check the different Event Hub limits, review Azure Event Hubs quota and limits.

Monitoring created resources

You can monitor the resources created by the streaming API using Azure Monitor. For more information, see Log Analytics workspace data export in Azure Monitor.

Tip

Do you want to learn more? Engage with the Microsoft Security community in our Tech Community: Microsoft Defender XDR Tech Community.