Efficient Way to read the custom log files from azure blob storage to be ingested to sentinel

Question

We have a application writing logs into the azure blob storage. We need a step by step guide to read the logs from blob storage and ingest them to sentinel SIEM tool

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

Step 1: Set Up Prerequisites

Ensure the following are set up:

1. Azure Storage Account: Your application logs are stored in an Azure Blob Storage container.
2. Microsoft Sentinel: Installed and running in your Azure Log Analytics workspace.
3. Permissions:
   • Ensure the Managed Identity or Service Principal accessing the blob storage has Storage Blob Data Reader role.
     • The Log Analytics Contributor role is required to write to Sentinel.

---

Step 2: Choose an Ingestion Method

There are three primary methods to ingest logs from Azure Blob Storage into Sentinel:

1. Using Azure Sentinel’s Data Connectors (Best for structured logs)
2. Using Azure Logic Apps (Best for event-driven log ingestion)
3. Using an Azure Function (Best for custom parsing and transformation)

---

Step 3: Implement the Chosen Approach

Option 1: Use Azure Sentinel’s Blob Storage Data Connector

If your logs are structured and supported by Sentinel's builtin Azure Blob Storage connector, use this:

1. Open Microsoft Sentinel.
2. Go to Data Connectors and search for Azure Blob Storage.
3. Click Open Connector Page and follow these steps:
   • Grant Sentinel access to the Azure Storage Account.
     • Define the log structure and format.
       • Map the blob data to Log Analytics tables.

Limitations: This method supports specific log formats. If your logs are in JSON, CSV, or syslog, it may work well, but unstructured logs might need preprocessing.

1. If logs are structured (JSON, CSV), use Sentinel’s built-in connector.
2. If event-driven ingestion is needed, use Azure Logic Apps.
3. If custom parsing or transformation is required, AzFunc is the best choice.

Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.