This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 87%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Hello,
I am experiencing an issue with the following configuration of my Data Collection Rule (DCR):
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"dataCollectionRules_name": {
"defaultValue": "dcr-integration",
"type": "String"
},
"logAnalyticsWorkspaceId": {
"defaultValue": "/subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/resourceGroups/my-resource-group/providers/microsoft.operationalinsights/workspaces/my-log-workspace",
"type": "String"
}
},
"resources": [
{
"type": "Microsoft.Insights/dataCollectionRules",
"apiVersion": "2023-03-11",
"name": "[parameters('dataCollectionRules_name')]",
"location": "francecentral",
"kind": "WorkspaceTransforms",
"properties": {
"dataSources": {},
"destinations": {
"logAnalytics": [
{
"workspaceResourceId": "[parameters('logAnalyticsWorkspaceId')]",
"name": "log-destination"
}
]
},
"dataFlows": [
{
"streams": [
"Microsoft-Table-ContainerLogV2"
],
"destinations": [
"log-destination"
],
"transformKql": "source\n| where tostring(LogMessage) !contains \"Picked up _JAVA_OPTIONS\"\n| where tostring(LogMessage) !contains \"Spring Boot\"\n"
}
]
}
}
]
}
With this configuration, I expect logs containing "Spring Boot" or "_JAVA_OPTIONS" to be filtered out and not appear in the ContainerLogV2 table. However, these logs are still being ingested.
I also tried modifying my DCR to use the Microsoft-ContainerLogV2 table instead, but this did not change anything.
My issue is simple: logs from my AKS cluster are being sent to ContainerLogV2, and I want to filter out specific messages so they are no longer collected.
Could you help me understand why these filters are not working and how I can correctly apply them?
Thank you in advance for your support.
Best regards, Melvin
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 51%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESE%3C/text%3E%3C/svg%3E)
Hi Masdieu, Melvin,
Good catch! Glad the issue is resolved for you finally. I will have this answer promoted by reposting it. As an Original Poster will not be able to accept your own answer. This is in the attempt to help others looking for a solution for a similar issue.
{
"properties": {
"immutableId": "dcr-02d318a9feed4f47b40c926a112fe692",
"dataSources": {
"extensions": [
{
"streams": [
"Microsoft-Perf",
"Microsoft-InsightsMetrics",
"Microsoft-ContainerLog",
"Microsoft-ContainerLogV2",
"Microsoft-KubeEvents",
"Microsoft-KubePodInventory",
"Microsoft-ContainerInventory",
"Microsoft-ContainerNodeInventory",
"Microsoft-KubeNodeInventory",
"Microsoft-KubeServices",
"Microsoft-KubePVInventory"
],
"extensionName": "ContainerInsights",
"extensionSettings": {
"dataCollectionSettings": {
"enableContainerLogV2": true,
"interval": "5m",
"namespaceFilteringMode": "Exclude",
"namespaces": [
"kube-system",
"gatekeeper-system",
"keda",
"calico-system",
"tigera-operator",
"snowsat-bot",
"cert-manager"
]
}
},
"inputDataSources": [],
"name": "ContainerInsightsExtension"
}
]
},
"destinations": {
"logAnalytics": [
{
"workspaceResourceId": "/subscriptions/subscription_id/resourceGroups/snowsat-rg-cluster-production/providers/Microsoft.OperationalInsights/workspaces/snowsat-log-production",
"workspaceId": "xxx",
"name": "2074f46d40fc414d872f44d15c1c86e3"
}
]
},
"dataFlows": [
{
"streams": [
"Microsoft-ContainerLogV2"
],
"destinations": [
"2074f46d40fc414d872f44d15c1c86e3"
],
"transformKql": "source | where LogMessage !has \"Picked up _JAVA_OPTIONS\" and LogMessage !has \"Spring Boot\""
},
{
"streams": [
"Microsoft-Perf",
"Microsoft-InsightsMetrics",
"Microsoft-ContainerLog",
"Microsoft-KubeEvents",
"Microsoft-KubePodInventory",
"Microsoft-ContainerInventory",
"Microsoft-ContainerNodeInventory",
"Microsoft-KubeNodeInventory",
"Microsoft-KubeServices",
"Microsoft-KubePVInventory"
],
"destinations": [
"2074f46d40fc414d872f44d15c1c86e3"
]
}
],
"provisioningState": "Succeeded"
},
"location": "francecentral",
"name": "MSCI-francecentral-k8s-production",
"apiVersion": "2022-06-01"
}
Thanks again for sharing the solution here. Have a good day!
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 4%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Hello Masdieu, Melvin,
Welcome to the Microsoft Q&A and thank you for posting your questions here.
I understand that your Data Collection Rule (DCR) is not filtering out specific log messages from ContainerLogV2 as expected.
Logs are not filtered because transformKql applies after ingestion and the correct approach is to apply filtering at the dataSources level.
To resolve this, and to ensures logs containing Spring Boot or _JAVA_OPTIONS are never collected you can do the followings:
ContainerInsights, confirm that the stream in the DCR should be Microsoft-ContainerLogV2 instead of Microsoft-Table-ContainerLogV2. - https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-log-query something like this:
ContainerLogV2
| summarize count() by SourceSystem
ContainerLogV2
| where tostring(LogMessage) !contains "Picked up _JAVA_OPTIONS"
| where tostring(LogMessage) !contains "Spring Boot"
LogMessage is a string field. If not, cast it explicitly:
| extend LogMessage = tostring(LogMessage)
"dataSources": {
"extensions": [
{
"name": "container-logs",
"streams": ["Microsoft-ContainerLogV2"],
"extensionName": "ContainerInsights",
"extensionParameters": {
"filtering": {
"filter": "tostring(LogMessage) !contains \"Picked up _JAVA_OPTIONS\" and tostring(LogMessage) !contains \"Spring Boot\""
}
}
}
]
}
I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.
Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.
Hi Masdieu, Melvin,
Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.
Just checking in to see if you had a chance to review the answer posted by Sina Salam on your question.
We understand from your query that you are experiencing an issue while trying to create DCR rule to get exception of Java and Spring boot.
DCR metrics are collected automatically for all DCRs, and you can analyze them using metrics explorer like the platform metrics for other Azure resources. Input stream is included as a dimension so if you have a DCR with multiple input streams, you can analyze each by filtering or splitting.
| where type in~ ('microsoft.insights/scheduledqueryrules') and ['kind'] !in~ ('LogToMetric')
| extend severity = strcat("Sev", properties["severity"])
| extend enabled = tobool(properties["enabled"])
| where enabled in~ ('true')
| where tolower(properties["targetResourceTypes"]) matches regex 'microsoft.operationalinsights/workspaces($|/.*)?' or tolower(properties["targetResourceType"]) matches regex 'microsoft.operationalinsights/workspaces($|/.*)?' or tolower(properties["scopes"]) matches regex 'providers/microsoft.operationalinsights/workspaces($|/.*)?'
| where properties contains "Perf" or properties contains "InsightsMetrics" or properties contains "ContainerInventory" or properties contains "ContainerNodeInventory" or properties contains "KubeNodeInventory" or properties contains"KubePodInventory" or properties contains "KubePVInventory" or properties contains "KubeServices" or properties contains "KubeEvents"
| project id,name,type,properties,enabled,severity,subscriptionId
| order by tolower(name) asc
Retrieve all error logs for a particular DCR
DCRLogErrors
| where _ResourceId == "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/my-resource-group/providers/microsoft.insights/datacollectionrules/my-dcr"
Please follow the below document for your reference:
If you found it helpful, could you kindly click the “Accept and upvote” on the post.
If you have any further queries, please let us know we are glad to help you.
Hello @Sina Salam
I've done some tests and modifications, thank you for the clarification about the transormation.
I have updated the configuration as follows :
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"dataCollectionRules_snowsat_dcr_integration_name": {
"defaultValue": "dcr-integration",
"type": "String"
},
"workspaces_snowsat_log_integration_externalid": {
"defaultValue": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xxxxxxxx-rg-integration/providers/Microsoft.OperationalInsights/workspaces/xxxxxxxx-log-integration",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Insights/dataCollectionRules",
"apiVersion": "2023-03-11",
"name": "[parameters('dataCollectionRules_snowsat_dcr_integration_name')]",
"location": "francecentral",
"kind": "Linux",
"properties": {
"dataSources": {
"extensions": [
{
"streams": [
"Microsoft-ContainerLogV2"
],
"extensionName": "ContainerInsights",
"extensionSettings": {
"filtering": {
"filter": "tostring(LogMessage) !contains \"Picked up _JAVA_OPTIONS\" and tostring(LogMessage) !contains \"Spring Boot\""
}
},
"inputDataSources": [],
"name": "container-logs"
}
]
},
"destinations": {
"logAnalytics": [
{
"workspaceResourceId": "[parameters('workspaces_snowsat_log_integration_externalid')]",
"name": "la-workspace"
}
]
},
"dataFlows": [
{
"streams": [
"Microsoft-ContainerLogV2"
],
"destinations": [
"la-workspace"
]
}
]
}
}
]
}
Unfortunately, this does not resolve the issue, and I am still seeing logs in containerlogv2.
However, I noticed that another DCR has been automatically created with the following configuration :
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"dataCollectionRules_MSCI_francecentral_snowsat_k8s_integration_name": {
"defaultValue": "xxxxxxxx-francecentral-xxxxxxxx-integration",
"type": "String"
},
"workspaces_snowsat_log_integration_externalid": {
"defaultValue": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xxxxxxxx-rg-integration/providers/Microsoft.OperationalInsights/workspaces/xxxxxxxx-log-integration",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Insights/dataCollectionRules",
"apiVersion": "2023-03-11",
"name": "[parameters('dataCollectionRules_MSCI_francecentral_snowsat_k8s_integration_name')]",
"location": "francecentral",
"kind": "Linux",
"properties": {
"dataSources": {
"syslog": [],
"extensions": [
{
"streams": [
"Microsoft-ContainerInsights-Group-Default"
],
"extensionName": "ContainerInsights",
"extensionSettings": {
"dataCollectionSettings": {
"interval": "5m",
"namespaceFilteringMode": "Exclude",
"namespaces": [
"kube-system",
"gatekeeper-system",
"keda",
"calico-system",
"tigera-operator"
],
"enableContainerLogV2": true
}
},
"name": "ContainerInsightsExtension"
}
]
},
"destinations": {
"logAnalytics": [
{
"workspaceResourceId": "[parameters('workspaces_snowsat_log_integration_externalid')]",
"name": "la-workspace"
}
]
},
"dataFlows": [
{
"streams": [
"Microsoft-ContainerInsights-Group-Default"
],
"destinations": [
"la-workspace"
]
}
]
}
}
]
}
Could there be a conflict between these two configurations? or am I missing something in the first configuration?
Thank you for your help !
Melvin
Hello Masdieu, Melvin,
Thanks for your feedback.
There is misconfiguration in extension parameters. The filter was placed under extensionSettings instead of extensionParameters see Microsoft Docs - https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-data-collection-filter#configure-log-filtering
Also, the second, automatically generated DCR (using Microsoft-ContainerInsights-Group-Default) is likely collecting unfiltered logs, overriding your custom DCR.
STEPS:
dataSources section to use extensionParameters (not extensionSettings):
"dataSources": {
"extensions": [
{
"streams": ["Microsoft-ContainerLogV2"],
"extensionName": "ContainerInsights",
"extensionParameters": { // Correct parameter name
"filtering": {
"filter": "tostring(LogMessage) !contains 'Picked up _JAVA_OPTIONS' and tostring(LogMessage) !contains 'Spring Boot'"
}
},
"name": "container-logs"
}
]
}
Microsoft-ContainerInsights-Group-Default) is managed by Azure Monitor for Containers, disable it by:
Success
Hello @Sina Salam
Thank you for the details, I tried almost everything, but it's not working...
I created the DCR with terraform, and it create the resource with extensionSettings
data_sources {
extension {
extension_name = "ContainerInsights"
name = "container-logs"
streams = ["Microsoft-ContainerLogV2"]
extension_json = jsonencode({
filtering = {
filter = "tostring(LogMessage) !contains \"Picked up _JAVA_OPTIONS\" and tostring(LogMessage) !contains \"Spring Boot\""
}
})
}
}
About container insights, we use it a lot, I'm not sure about disabled it. We will lose all of the analysis it provides.
And when I tried to overwrite the automatic DCR with the filter, it allow me everything but doesn't take the filter in his configuration. The configuration returned by azure cli is that :
There is everything but not the extensionParameters
Data source sent :
"dataSources": {
"extensions": [
{
"extensionName": "ContainerInsights",
"extensionParameters": {
"filtering": {
"filter": "tostring(LogMessage) !contains 'Picked up _JAVA_OPTIONS' and tostring(LogMessage) !contains 'Spring Boot'"
}
},
"extensionSettings": {
"dataCollectionSettings": {
"enableContainerLogV2": true,
"interval": "5m",
"namespaceFilteringMode": "Exclude",
"namespaces": [
"kube-system",
"gatekeeper-system",
"keda",
"calico-system",
"tigera-operator"
]
}
},
"name": "ContainerInsightsExtension",
"streams": [
"Microsoft-ContainerInsights-Group-Default",
"Microsoft-ContainerLogV2"
]
}
],
"syslog": []
}
Data source got from azure cli when command has been validated
{
"dataSources": {
"extensions": [
{
"extensionName": "ContainerInsights",
"extensionSettings": {
"dataCollectionSettings": {
"enableContainerLogV2": true,
"interval": "5m",
"namespaceFilteringMode": "Exclude",
"namespaces": [
"kube-system",
"gatekeeper-system",
"keda",
"calico-system",
"tigera-operator"
]
}
},
"name": "ContainerInsightsExtension",
"streams": [
"Microsoft-ContainerInsights-Group-Default",
"Microsoft-ContainerLogV2"
]
}
],
"syslog": []
}
}
Thank you for your help.
Hope we will find a solution
Melvin
Hello Masdieu, Melvin,
Thanks always for detailed feedback.
There is something you're not just doing right. This reply is to clear the air and know what is right to get your goals.
Follow these guides:
transformKql after ingestion. extensionSettings instead of extensionParameters. # Check if the auto-generated DCR is active:
az monitor data-collection rule list --resource-group <resource-group-name>
# The command above lists all data collection rules (DCRs) in the specified # resource group. Look for the `Microsoft-ContainerInsights-Group-Default`
# DCR in the output.
# If `Microsoft-ContainerInsights-Group-Default` is active and unfiltered,
# disable the default AKS monitoring:
az aks disable-addons --addons monitoring --name <aks-cluster-name> --resource-group <resource-group-name>
--addons instead of -a for better clarity and consistency with Azure CLI documentation - https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-associations and https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-overviewextensionParameters (Not extensionSettings) Correct JSON placement:
"dataSources": {
"extensions": [
{
"streams": ["Microsoft-ContainerLogV2"],
"extensionName": "ContainerInsights",
"extensionParameters": {
"filtering": {
"filter": "tostring(LogMessage) !contains \"Picked up _JAVA_OPTIONS\" and tostring(LogMessage) !contains \"Spring Boot\""
}
},
"name": "container-logs"
}
]
}
# List all active DCRs applied to the AKS cluster:
az monitor data-collection rule list --resource-group <your-resource-group> --output table
# If an automatically created DCR (`Microsoft-ContainerInsights-Group- Default`)
# is collecting logs without filtering , it must be modified or disabled:
az monitor data-collection rule delete --name <auto-generated-dcr-name> --resource-group <your-resource-group>
ContainerLogV2
| where tostring(LogMessage) contains "Picked up _JAVA_OPTIONS" or tostring(LogMessage) contains "Spring Boot"
extensionParameters instead of extensionSettings:
"dataSources": {
"extensions": [
{
"name": "container-logs",
"streams": ["Microsoft-ContainerLogV2"],
"extensionName": "ContainerInsights",
"extensionParameters": {
"filtering": {
"filter": "tostring(LogMessage) !contains \"Picked up _JAVA_OPTIONS\" and tostring(LogMessage) !contains \"Spring Boot\""
}
}
}
]
}
FINALY NOTE THE FOLLOWINGS:
Goodluck
Hi Masdieu, Melvin,
I have provided more details, hope it helps.
create custom KQL queries to exclude logs containing "Spring Boot" or "_JAVA_OPTIONS". Here's an example query to exclude these logs:
ContainerLogV2
| where not (Message contains "Spring Boot" or Message contains "_JAVA_OPTIONS")
{
"filters": [
{
"filter": "grep",
"exclude": {
"key": "message",
"pattern": "(Spring Boot|_JAVA_OPTIONS)"
}
}
],
"output": {
"match": "container.*",
"type": "azuremonitor",
"customer_id": "YOUR_WORKSPACE_ID",
"shared_key": "YOUR_SHARED_KEY",
"log_type": "ContainerLog"
}
}
logging.level.org.springframework.boot=OFF
Modify the logback.xml or log4j2.xml to exclude certain messages.
Example in logback to exclude messages containing "_JAVA_OPTIONS":
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss} - %msg%n</pattern>
</encoder>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>INFO</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
Please follow the below document for your reference:
If you found it helpful, could you kindly click the “Accept and upvote” on the post.
If you have any further queries, please let us know we are glad to help you.
Hi Masdieu, Melvin,
Just checking in to see if you had a chance to review the updated answer on your question. Please let us know if it was helpful and feel free to reach out if you have any further queries.
If you found the information useful, please click "Upvote" on the post to let us know.
Hello everyone,
Here is the solution that permit me to do what I want to :
{
"properties": {
"immutableId": "dcr-02d318a9feed4f47b40c926a112fe692",
"dataSources": {
"extensions": [
{
"streams": [
"Microsoft-Perf",
"Microsoft-InsightsMetrics",
"Microsoft-ContainerLog",
"Microsoft-ContainerLogV2",
"Microsoft-KubeEvents",
"Microsoft-KubePodInventory",
"Microsoft-ContainerInventory",
"Microsoft-ContainerNodeInventory",
"Microsoft-KubeNodeInventory",
"Microsoft-KubeServices",
"Microsoft-KubePVInventory"
],
"extensionName": "ContainerInsights",
"extensionSettings": {
"dataCollectionSettings": {
"enableContainerLogV2": true,
"interval": "5m",
"namespaceFilteringMode": "Exclude",
"namespaces": [
"kube-system",
"gatekeeper-system",
"keda",
"calico-system",
"tigera-operator",
"snowsat-bot",
"cert-manager"
]
}
},
"inputDataSources": [],
"name": "ContainerInsightsExtension"
}
]
},
"destinations": {
"logAnalytics": [
{
"workspaceResourceId": "/subscriptions/subscription_id/resourceGroups/snowsat-rg-cluster-production/providers/Microsoft.OperationalInsights/workspaces/snowsat-log-production",
"workspaceId": "xxx",
"name": "2074f46d40fc414d872f44d15c1c86e3"
}
]
},
"dataFlows": [
{
"streams": [
"Microsoft-ContainerLogV2"
],
"destinations": [
"2074f46d40fc414d872f44d15c1c86e3"
],
"transformKql": "source | where LogMessage !has \"Picked up _JAVA_OPTIONS\" and LogMessage !has \"Spring Boot\""
},
{
"streams": [
"Microsoft-Perf",
"Microsoft-InsightsMetrics",
"Microsoft-ContainerLog",
"Microsoft-KubeEvents",
"Microsoft-KubePodInventory",
"Microsoft-ContainerInventory",
"Microsoft-ContainerNodeInventory",
"Microsoft-KubeNodeInventory",
"Microsoft-KubeServices",
"Microsoft-KubePVInventory"
],
"destinations": [
"2074f46d40fc414d872f44d15c1c86e3"
]
}
],
"provisioningState": "Succeeded"
},
"location": "francecentral",
"name": "MSCI-francecentral-k8s-production",
"apiVersion": "2022-06-01"
}
Nothing else was working...
The extensionparameter is not existing.
I must use this transformation in the automatic DCR
Thank you for your help
Hello Masdieu, Melvin,
Thank you for letting me know. For the sake of those who read from the top.
Modifying the automatic DCR to include the transformKql filter, ensuring logs containing "Spring Boot" or "_JAVA_OPTIONS" were excluded:
"dataFlows": [
{
"streams": ["Microsoft-ContainerLogV2"],
"destinations": ["la-workspace"],
"transformKql": "source | where LogMessage !has \"Picked up _JAVA_OPTIONS\" and LogMessage !has \"Spring Boot\""
}
]
This worked because:
Yes, I did advise modifying the automatic DCR (partial changes) to include filtering at the dataSources level but did not anticipate Azure’s limitations in accepting extensionParameters for automatic DCRs. Your workaround with transformKql was a pragmatic deviation from my advised and that is a good one.
NOTE: I have given you a thump-up. I will post another constructive answer to close this thread and create a more meaningful easy access for your answer.
Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer. Accepted answers show up at the top, resulting in improved discoverability for others.
Thank you for letting us know once again.
Hello Masdieu, Melvin,
I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!
Issue: Customer logs from my AKS cluster are being sent to ContainerLogV2, and I want to filter out specific messages, so they are no longer collected.
Error Message: No specific error - Customer have the configurations has explained in the problem statement in the question above. Expected the logs containing "Spring Boot" or "_JAVA_OPTIONS" to be filtered out and not appear in the ContainerLogV2 table. However, these logs are still being ingested. Customer also tried modifying DCR to use the Microsoft-ContainerLogV2 table instead, but noting was changed.
Solution: Customer found a workaround solution below because the extension recommended or anything else could not work:
{
"properties": {
"immutableId": "dcr-02d318a9feed4f47b40c926a112fe692",
"dataSources": {
"extensions": [
{
"streams": [
"Microsoft-Perf",
"Microsoft-InsightsMetrics",
"Microsoft-ContainerLog",
"Microsoft-ContainerLogV2",
"Microsoft-KubeEvents",
"Microsoft-KubePodInventory",
"Microsoft-ContainerInventory",
"Microsoft-ContainerNodeInventory",
"Microsoft-KubeNodeInventory",
"Microsoft-KubeServices",
"Microsoft-KubePVInventory"
],
"extensionName": "ContainerInsights",
"extensionSettings": {
"dataCollectionSettings": {
"enableContainerLogV2": true,
"interval": "5m",
"namespaceFilteringMode": "Exclude",
"namespaces": [
"kube-system",
"gatekeeper-system",
"keda",
"calico-system",
"tigera-operator",
"snowsat-bot",
"cert-manager"
]
}
},
"inputDataSources": [],
"name": "ContainerInsightsExtension"
}
]
},
"destinations": {
"logAnalytics": [
{
"workspaceResourceId": "/subscriptions/subscription_id/resourceGroups/snowsat-rg-cluster-production/providers/Microsoft.OperationalInsights/workspaces/snowsat-log-production",
"workspaceId": "xxx",
"name": "2074f46d40fc414d872f44d15c1c86e3"
}
]
},
"dataFlows": [
{
"streams": [
"Microsoft-ContainerLogV2"
],
"destinations": [
"2074f46d40fc414d872f44d15c1c86e3"
],
"transformKql": "source | where LogMessage !has \"Picked up _JAVA_OPTIONS\" and LogMessage !has \"Spring Boot\""
},
{
"streams": [
"Microsoft-Perf",
"Microsoft-InsightsMetrics",
"Microsoft-ContainerLog",
"Microsoft-KubeEvents",
"Microsoft-KubePodInventory",
"Microsoft-ContainerInventory",
"Microsoft-ContainerNodeInventory",
"Microsoft-KubeNodeInventory",
"Microsoft-KubeServices",
"Microsoft-KubePVInventory"
],
"destinations": [
"2074f46d40fc414d872f44d15c1c86e3"
]
}
],
"provisioningState": "Succeeded"
},
"location": "francecentral",
"name": "MSCI-francecentral-k8s-production",
"apiVersion": "2022-06-01"
}
I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.
Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.