Azure DiagnosticMonitor - Custom Logs

Azure Diagnostics custom logs are not moving to the storage account.  I've configured the diagnostics as follows for custom logs.

Code in OnStart method:

// Get the default initial configuration for DiagnosticMonitor.

DiagnosticMonitorConfiguration diagnosticConfiguration = DiagnosticMonitor.GetDefaultInitialConfiguration();

// Set scheduled transfer interval for infrastructure logs to 1 minute

diagnosticConfiguration.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(10);

// Specify a logging level to filter records to transfer

diagnosticConfiguration.DiagnosticInfrastructureLogs.ScheduledTransferLogLevelFilter = LogLevel.Error;

// Create a new DirectoryConfiguration object.

DirectoryConfiguration directoryConfiguration = new DirectoryConfiguration();

// Add the name for the blob container in Windows Azure storage.

directoryConfiguration.Container = "wad-custom-logs";

// Add the directory size quota.

directoryConfiguration.DirectoryQuotaInMB = 2048;

// Add the log path for the role using RoleEnvironment.GetLocalResource().

directoryConfiguration.Path = RoleEnvironment.GetLocalResource("MyCustomLogs").RootPath;

// Add the directoryConfiguration to the Directories collection.

diagnosticConfiguration.Directories.DataSources.Add(directoryConfiguration);

// Schedule a transfer period of 5 minutes.

diagnosticConfiguration.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(5.0);

diagnosticConfiguration.Logs.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(10);

// Start the DiagnosticMonitor using the diagnosticConfig and our connection string.

DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", diagnosticConfiguration);

 

Configuration in Service Definition file:

<LocalResources>

<LocalStorage name="DiagnosticStore" sizeInMB="8192" cleanOnRoleRecycle="false"/>

<LocalStorage name="MyCustomLogs" sizeInMB="2048" cleanOnRoleRecycle="false" />

</LocalResources>

 

ServiceConfiguraiton.csfg content:

<ServiceConfiguration serviceName="CustomLogDiagnostics" xmlns="https://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1" osVersion="*">

<Role name="WebRole1">

<Instances count="1" />

<ConfigurationSettings>

<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=wcftracesample;AccountKey=QW340M3uUVGc3Y9CEkg2hZ1VIhuWGX8ZLbaDY97FJI+tPguwrFlpmf5i13Rq2Mcb1J1SeIfUs4ce9PQc4/WA3w==" />

<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />

<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="RemoteUser" />

<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="MIIBnQYJKoZIhvcNAQcDoIIBjjCCAYoCAQAxggFOMIIBSgIBADAyMB4xHDAaBgNVBAMME1dpbmRvd3MgQXp1cmUgVG9vbHMCEEVj/zzTXtqlSuKR51zsoVswDQYJKoZIhvcNAQEBBQAEggEAOLeYLcBlAX88/9SU1Z2b7wlgGd6AWqwU7I4t7Lv4KTEhNNLrnqrshNSSbqISRVpWycx66dbMoJUJ2nxAvyN0TFt0uocAjivnFU1/rUvaSCaBXOCUhHe4OfHh1wUSRjzMn/H177y0elMC+/AwFILbMwakVxgFrqCFBroE6K9CpD1yYH7q9Ic9YMGAukRo7HgOJ5nq24hfCZ0G2CyT5mEMb09j6/4IhtYiYNsVYNSpaq+bQdvrXEEzRIVdMdZsAtNfP/2gFCt/tTHcavsoOeHZYgZ3GIeo+bihOKFRJTOWsECyPNr9fgFR9TeDdK8IOU42IyBmhUHvTVeVnf8YbmVmuzAzBgkqhkiG9w0BBwEwFAYIKoZIhvcNAwcECNHwIT0gVndfgBA9CD+sBmMV0EcGMdwVaqn/" />

<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2011-08-22T23:59:59.0000000-05:00" />

<Setting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" />

</ConfigurationSettings>

<Certificates>

<Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="C8CB8231FA78C369D48BC23F690EC23A97CBD815" thumbprintAlgorithm="sha1" />

</Certificates>

</Role>

</ServiceConfiguration>

As you can see above, everything is configured correct but the logs do not move to the storage account.

-   Through some research, I was able to find that there are some permission issues in SDK 1.3 and 1.4 which would not let the diagnostics read the logs and move it to storage account.

-   I used CACLS in the startup task to resolve the issue, basically gave everyone full rights on Resources folder.

CACLS C:\Resources /E /T /C /G "Everyone":F

CACLS in startup task can be used as a workaround until the problem is resolved in future versions of Azure SDK.