Share via

Security logging and analysis options in Azure

  • Article

imageToday we have a great blog post for those of you interested in the security logging options in Microsoft Azure. Sanjay Narang was the thought leader on this article, with Becky Ochs, Namgyal Dolker and I working together with him. Enjoy!

We’ve seen that many of you are looking for a way to collect and analyze security (and other) logs in Azure. Analysis of these logs can provide you with a ton of useful information and insight into your current security posture. Armed with this information, you’ll be able to detect security issues in your Azure environment and be able to take action based on that information.

For example, you can use these logs to:

  • Detect threats and policy violations
  • Help achieve regulatory compliance
  • Search for anomalous network, host, or user activity

Security Information and Event Management (SIEM) systems are used to collect and analyze logs and many of you are using them today to help you with your log collection and analysis. However, to get the most out of your SIEM systems, you must be able to get the logging information you need into them, and that means knowing what logging information is available to you and how you can access it.

To help you solve this problem, this blog post will provide you with a summary of the different types of logs that are available today in Azure and describe how to access those logs. We’ll also provide links to REST APIs for accessing each type of log. REST APIs enable you to import these logs into most SIEM engines.

Subscription Management Level Logs

Azure can log the events that impact your entire subscription or you can monitor events that impact specific resources.

Event Services Logs

Azure provides the Azure Insights REST API as well as .Net Library APIs. You can use these APIs to get visibility into your Azure resources. All task operations conform to the HTTP/1.1 protocol specification and each operation returns an x-ms-request-id header that can be used to obtain information about the request.

A key part of these APIs is Event Services. Event Services enables you to obtain information about events (or operations) related to your Azure resources. Logs of all these events/operations are exposed through the well-documented Management Events REST API for the subscription. The Management Events Rest API logs all Create/Update/Delete operations, as well as privileged Read operations. 

Event Services is used by portals to enable you to easily view logs in a user interface. It is also consumed by other APIs, such as the Azure Resource Manager (ARM) API.

  • Operations Logs: these are available in the v1 portal under management services. You can also find them through the service management Operations REST API.
  • Audit Logs: these are available in the new portal and through PowerShell commands such as Get-AzureSubscriptionIdLog or Get-AzureResourceProviderLog.

Service Level Logs

In addition to subscription level logs, many services provide more detailed logs.

Diagnostics Logs

Diagnostics logs can be used for Virtual Machines and Cloud Services. These logs include:

  • Event Logs
  • Custom Application Logs
  • IIS Logs
  • Crash Dumps
  • Performance Counters

All of these logs are described in detail in the article Microsoft Azure Security and Audit Log Management.

Since diagnostics logs are stored in Azure Storage, you can use Storage Service REST API to pull these logs into most SIEM engines. The collection can be scheduled at regular intervals by configuring the ScheduledTransferPeriod Property. The minimum transfer period you can specify is 1 minute.

Azure AD Reports

Azure AD Reports provide a variety of activity, security and audit reports. You can use this information to determine where security risks may lie. This helps guide your security investigations and subsequent mitigations. Reports are also available through the REST API.

For more information on what is included in the audit reports, see Azure Active Directory Audit Report Events.

Azure Storage Analytics

Azure Storage Analytics provides logs (traces of executed requests for Blobs, Tables and Queues) and metrics (summary of key capacity and request statistics). Logs can be fetched using the Storage Service REST API.

From a security perspective, Azure Storage Analytics allows you to run the following the types of analyses (more details here):

  • How many anonymous requests is my application seeing from a given range of IP addresses?
  • Which containers are being accessed the most?
  • How many times is a particular SAS URL being accessed and how?
  • Who issued the request to delete a container?

Azure SQL Database Auditing

Azure SQL Database Auditing tracks database events and writes audited events to an audit log in your Azure Storage account. More details are available here. These logs can be fetched using Storage Service REST API.

Next Step

The article Microsoft Azure Security and Audit Log Management provides detailed information about logging and audit management in Azure. We highly recommend that you read the article to get further insights into much of what we covered in this blog post.

We hope that you find these logging and auditing resources useful. If you have questions, comments, or suggestions, please let us know! Just enter a comment at the bottom of the blog and we’ll answer your questions as quickly as possible.

Thanks!
Tom

Tom Shinder
Project Manager, Azure Security Engineering
@tshinder | Facebook | LinkedIn | Email | Web | Bing me! | GOOG me!

image

Comments

  • Anonymous
    September 19, 2015
    Thanks, looking forward to explore further.

  • Anonymous
    November 23, 2015
    Great reference!

  • Anonymous
    December 02, 2015
    Thanks!!! Let us know if you have any questions.

  • Anonymous
    January 19, 2016
    I just want to say that I like your posting. In fact I am using your site regularly. Your articles are very effective and i am very thankful to you for sharing this site with knowledgeable content .

  • Anonymous
    February 07, 2016
    Please let me know if we can have the logs collected for PaaS instances ( Azure web app deployed in ASE)

  • Anonymous
    February 07, 2016
    Thanks O2 systems!

  • Anonymous
    February 07, 2016
    Hi Ashvini - Just to be clear, which you say "ASE" do you mean "ASC"? (Azure Security Center)? Thanks! Tom

    • Anonymous
      March 07, 2016
      I believe he is referring to App Service Environment (ASE). But since you brought it up ;-). Are there APIs that can be leveraged to get to the Azure Security Center logs?
      • Anonymous
        March 10, 2016
        Hi Peter -Good idea! :) - not yet, but keep your eyes on this blog and we'll let you know when that happens.Thanks!Tom