Guidance on Filtering AppTraces Logs to Optimize Sentinel Workspace Usage

Someiah C S 80

Hi Community,

I'm seeking advice on how to filter out AppTraces logs from being ingested into our Sentinel workspace. These logs are consuming significant storage space and, being categorized under Analytics logs, are contributing to increased costs. Since we're not utilizing them for our security monitoring purposes, I'd like to exclude them from ingestion.

I understand that implementing data collection rules (DCRs) can help manage log ingestion. However, I'm uncertain about the specific steps to configure these rules to filter out AppTraces logs effectively. Additionally, I'm aware that certain tables, including AppTraces, can be configured for Basic Logs, which might offer a more cost-effective solution.

Could anyone provide detailed guidance or share best practices on setting up these configurations? Any insights or resources would be greatly appreciated.

Thank you in advance for your assistance.

Sakshi Devkante 1,240 Reputation points Microsoft Vendor

2025-02-27T11:09:27.95+00:00

Hello Someiah,

Thank you for posting your query on Microsoft Q&A.

I understand that you want to filter out AppTraces logs from being ingested into your Azure Sentinel workspace. in this case you can use Data Collection Rules (DCRs) to exclude specific logs or configure Basic Logs for cost-effective storage.

Find out which data source is providing your Sentinel workspace with AppTraces logs. This might be a custom application or an Azure resource like an App Service.

A Data Collection Rule (DCR) should be created.
Navigate to the Azure portal's -> Azure Monitor service. Go to the Settings section and select -> Data Collection Rules. To create a new DCR, click Create.

Indicate which resources (such as particular App Services) you wish to exclude AppTraces logs from.

Add a Data Source and choose the relevant log type (such as Custom Logs or Azure Diagnostics) in the DCR. - Transformation is used to remove AppTraces logs.

For instance: Only logs with a Category that does not contain "AppTraces" are ingested thanks to this KQL (Kusto Query Language) adjustment. To use the filtering rule, connect the DCR to your Sentinel workspace.

Make sure that AppTraces logs are no longer being ingested by keeping an eye on your Sentinel workspace after applying the DCR.

AppTraces logs can be set up as "Basic Logs" if you still need to keep them but wish to cut expenses. Although basic logs are less expensive to keep, they have certain drawbacks, like shorter retention periods and delayed query availability. In your Sentinel workspace, identify the table where AppTraces logs are stored (e.g., AppTraces). Configure Basic Logs:

Go to your Log Analytics workspace in the Azure portal.

Navigate to ->Tables under the -> Settings section.

Locate the "AppTrace" table and click on it.

Change the table's configuration to "Basic Logs".

Ensure that the table is now set to Basic Logs, which will reduce storage costs.

Refer similar threads:
1.https://learn.microsoft.com/en-us/answers/questions/1851367/data-collection-rule-transformation-not-filtering

2.https://learn.microsoft.com/en-us/answers/questions/2086637/applying-custom-data-collection-rules-for-ama-to-r

3.https://learn.microsoft.com/en-us/answers/questions/2182885/issue-with-log-filtering-in-containerlogv2-using-d

Refer documents:
https://learn.microsoft.com/en-us/azure/sentinel/connect-data-sources?tabs=azure-portal
https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-overview
https://learn.microsoft.com/en-us/azure/sentinel/best-practices-data
https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-table-plans?tabs=portal-1

I hope this clarifies things.
Someiah C S 80 Reputation points

2025-02-27T12:29:07.7533333+00:00

Hi @Sakshi Devkante

Thanks for your response. The AppTraces logs are coming from Application Insights, so I’m aware of the source. However, when I tried to create a DCR, I wasn’t able to select Application Insights from the available resource types. The options listed below are selectable, but not Application Insights.

Is this due to a permission issue, or is it not possible to select Application Insights as a resource when configuring a DCR?
Ashok Gandhi Kotnana 3,895 Reputation points Microsoft Vendor

2025-02-27T15:56:40.3+00:00

Hi @Someiah C S ,

Adaptive Sampling automatically adjusts how much telemetry data is collected based on your app’s traffic. This helps reduce the amount of data sent to Application Insights, which can also lower the data sent to Sentinel. https://learn.microsoft.com/en-us/previous-versions/azure/azure-monitor/app/sampling

If possible, you can add custom logic to your application to log only important telemetry data. This allows you to filter out unnecessary logs, like App Traces, reducing the amount of data sent to Application Insights. https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/apptraces

In the Log Analytics workspace, you can use Kusto Query Language (KQL) to filter or modify data when running queries. While this helps improve query performance and can lower analysis costs, it doesn’t reduce ingestion costs since the data is still stored in the workspace. https://learn.microsoft.com/en-us/azure/azure-monitor/logs/queries?tabs=groupby

If you want to keep some telemetry data but reduce costs, you can set up Basic Logs for the relevant tables in your Log Analytics workspace. Basic Logs are cheaper to store, but they may offer less detail and slower query performance.

https://learn.microsoft.com/en-us/azure/azure-monitor/logs/cost-logs

If you found this answer helpful, please consider "upvote" it.

Share via

Guidance on Filtering AppTraces Logs to Optimize Sentinel Workspace Usage

Your answer