I want to Configure Azure Activity logs to stream to specified Storage account from all subscriptions, is there any Built In policy available which can be leveraged to send activity logs from all subscription to a pre-defined storage account.

Hi Kaushik Ray Welcome to Microsoft Q&A Forum! To stream Azure Activity logs to a specified storage account from all subscriptions, use a built-in policy that deploys diagnostic settings. Here are the steps for this configuration: 1.Azure offers a community policy called "Deploy Diagnostic Settings for Activity Log to storage account," which automatically configures diagnostic settings to stream Activity Log logs to a specified storage account whenever a subscription is created or updated without these settings. Key Features of the Policy: Name: Deploy Diagnostic Settings for Activity Log to storage account Effect: DeployIfNotExists Mode: All Description: Ensures that the diagnostic settings for Activity Logs are configured to stream to a designated storage account when any subscription is missing these settings 1.In the Azure portal, go to Storage accounts and create a new storage account to store the logs. 2.Use Azure Policy to assign the "Deploy Diagnostic Settings for Activity Log to storage account" policy across your subscriptions, ensuring activity logs are sent to the specified storage account. 3.After assigning the policy, verify that the diagnostic settings are applied by checking the Activity Log configuration in each subscription. 4.You can monitor and manage the activity logs stored in your designated storage account as per your requirements. Please let us know if you required anything

Configure Azure Activity logs to stream to specified Storage account from all subscriptions

Pavan Minukuri 1,045 Microsoft Vendor

Hi Kaushik Ray We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.

Kaushik Ray 0

What would be the hierarchy of the logs, also how will all the subscription logs be placed in the storage account, can we determine if it can be placed on a specific container like - 'activity-logs'

Kaushik Ray 0

I had the policy implemented, however I can see that the policy is failing compliance. None of the subscriptions(currently) have the diagnostic settings still they are not being created.

{
 
  "if": {
 
    "allOf": [
 
      {
 
        "field": "type",
 
        "equals": "Microsoft.Resources/subscriptions"
 
      }
 
    ]
 
  },
 
  "then": {
 
    "effect": "[parameters('effect')]",
 
    "details": {
 
      "type": "Microsoft.Insights/diagnosticSettings",
 
      "deploymentScope": "Subscription",
 
      "existenceScope": "Subscription",
 
      "name": "[parameters('profileName')]",
 
      "existenceCondition": {
 
        "allOf": [
 
          {
 
      "field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
 
      "equals": "[parameters('logsEnabled')]"
 
    },
 
          {
 
            "field": "Microsoft.Insights/diagnosticSettings/storageAccountId",
 
            "equals": "[parameters('storageAccountId')]"
 
          }
 
        ]
 
      },
 
      "roleDefinitionIds": [
 
        "/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293",
 
    "/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
 
    "/providers/microsoft.authorization/roleDefinitions/17d1049b-9a84-46fb-8f53-869881c3d3ab"
 
      ],
 
      "deployment": {
 
        "location": "westeurope",
 
        "properties": {
 
          "mode": "incremental",
 
          "template": {
 
            "$schema": "
 
            "contentVersion": "1.0.0.0",
 
            "parameters": {
 
              "storageAccountId": {
 
                "type": "string"
 
              },
 
              "profileName": {
 
                "type": "string"
 
              }
 
            },
 
            "variables": {},
 
            "resources": [
 
              {
 
                "type": "Microsoft.Insights/diagnosticSettings",
 
                "apiVersion": "2021-05-01-preview",
 
                "name": "[parameters('profileName')]",
 
                "location": "Global",
 
                "properties": {
 
                  "storageAccountId": "[parameters('storageAccountId')]",
 
                  "logs": [
 
                    {
 
                      "category": "Administrative",
 
                      "enabled": true
 
                    },
 
                    {
 
                      "category": "Security",
 
                      "enabled": true
 
                    },
 
                    {
 
                      "category": "Policy",
 
                      "enabled": true
 
                    }
 
                  ],
 
                  "metrics": []
 
                }
 
              }
 
            ],
 
            "outputs": {}
 
          },
 
          "parameters": {
 
            "storageAccountId": {
 
              "value": "[parameters('storageAccountId')]"
 
            },
 
            "profileName": {
 
              "value": "[parameters('profileName')]"
 
            }
 
          }
 
        }
 
      }
 
    }
 
  }
 
}

Pavan Minukuri 1,045 Microsoft Vendor

Hi Kaushik Ray
To address the issue of your Azure Policy failing compliance with diagnostic settings, consider the following steps based on your policy definition and current Azure capabilities.
1.Modify your existenceCondition to ensure it checks for all necessary log categories and their enabled states correctly. Here’s an example of how you might structure this:
"existenceCondition": {

"allOf": [

    {

        "count": {

            "field": "Microsoft.Insights/diagnosticSettings/logs[*]",

            "where": {

                "allOf": [

                    {

                        "field": "Microsoft.Insights/diagnosticSettings/logs[*].category",

                        "in": ["Administrative", "Security", "Policy"]

                    },

                    {

                        "field": "Microsoft.Insights/diagnosticSettings/logs[*].enabled",

                        "equals": true

                    }

                ]

            },

            "equals": 3

        }

    },

    {

        "field": "Microsoft.Insights/diagnosticSettings/storageAccountId",

        "equals": "[parameters('storageAccountId')]"

    }

]

}

2.Confirm that the role definitions in your policy grant enough permissions to create and manage diagnostic settings.
3.Makesure that the parameters being passed, such as storageAccountId and profileName, are valid and correspond to existing resources in your Azure
environment.
4.Use the Azure Policy compliance dashboard to check if the changes have resolved the compliance issues.
5.Review audit logs for any errors or warnings related to policy assignments or deployments.
6.Manually check a few subscriptions to see if diagnostic settings have been applied as expected.

Please let me know if you required anything!

Kaushik Ray 0

RI created remediation steps which picked 0 of 0 resources and didn't setup diagnostic settings, however after again creating remediation step then it created all diagnostics settings sending logs to storage account. In case in future there is a new subscription, then it should have the policy enforced and diagnostic settings enabled, I tried deleting one wokring subscription diagnostic settings and left it over the weekend, it evaluated to Compliance failed but didn't create the diagnostics settings. I think if I create the remediation again then it will, but that doesn't solve the purpose for newer subscriptions.

Looks for some suggestions.

Pavan Minukuri 1,045 Microsoft Vendor

@Kaushik Ray Sorry for delay!
The issue you're facing with Azure Policy and storage account diagnostics comes from how Azure evaluates compliance and handles remediation. Here's a simplified explanation:

Initial Remediation Steps: The first remediation didn’t apply correctly, but the second one did, setting up the diagnostic settings.

Compliance Evaluation: If diagnostic settings are deleted, Azure marks the resource as non-compliant, but it won’t automatically fix it unless you manually trigger remediation.

New Subscriptions: For new subscriptions, policies can enforce compliance automatically, but existing resources require manual remediation.

Policy Definition: Your policy must check configurations for all resource types and sub-resources to ensure full compliance.

Remediation Tasks: These are needed for immediate compliance, but any changes after deployment will require manual intervention.

Automatic Remediation Limitations: Azure doesn’t auto-fix existing resources unless a new remediation task is created.

Monitor Compliance: Regularly check compliance and trigger remediation tasks if needed.

Automation: Use Azure Automation or scripts to enforce compliance on existing resources.

Custom Policies: Create custom policies to ensure specific configurations are always checked.

Stay Updated: Keep up with Azure Policy updates for potential improvements in automatic remediation.

Refer: https://learn.microsoft.com/en-us/azure/governance/policy/how-to/remediate-resources?WT.mc_id=devops-9479-stmuraws&tabs=azure-portal
https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings-policy
https://github.com/Azure/azure-policy/issues/986

Please let me know if you required anything!

Share via

Configure Azure Activity logs to stream to specified Storage account from all subscriptions

1 answer

Your answer