Using Office 365 activity data to improve your Cybersecurity stance and capability
Overview and Contents
As an Office 365 customer and tenant administrator, you have access to a wide variety of user activity events logged from your Exchange Online, SharePoint Online and Azure Active Directory services. Office 365 provides detailed activity logs that can be acquired and analyzed to facilitate self-service forensic investigations to scope and remediate a suspected incident or breach as well as assist you to develop a basic system to manage information security and events.
This post will be the first of a series dedicated to helping you understand cybersecurity fundamentals and build a robust investigation capability within your Office 365 service. We will provide you with:
- The knowledge and tools to acquire your activity data from the Office 365 cloud
- Ingestion of the data into your choice of several platforms that fit your specific needs
- Perform a basic forensic investigation to identify and scope suspected malicious activity inside your tenancy
- Execute basic remediation within the service if you do identify a breach or malicious activity as part of your self-service investigation.
Subsequent posts will deal with discrete end-to-end breach scenarios. Some of the features we discuss here require specific Office 365 service level subscriptions. For information about availability, please see Office 365 Service Descriptions
Contents
- What does Cybersecurity mean in Office 365?
- How are my business assets identified within Office 365?
- How does this information increase my preparedness and response?
- Preparing for cybersecurity response and remediation within Office 365
- Review Permissions
- Enable Activity Logging
- Acquire the data!
- Set yourself up to analyze the data
- Analyze in flight
- Third party and Open Source analytical platforms
- Next Steps
- Blog Resources and Reference
- Additional Office 365 Information and Data Security Resources
What does Cybersecurity mean in Office 365?
First let’s talk about some cybersecurity specific language and how it applies to Office 365, the scope and goal of this post, and how this information relates directly to you and your tenancy.
You may have seen or heard the abbreviation PDIR lately with the prevalence of security incidents in the media. PDIR stands for Protect, Detect, Investigate and Remediate and is a fundamental process or methodology to plan for, react to and recover from a cybersecurity breach of your valuable data. In this post, we will be focusing building towards the Investigation and Remediate capabilities within your Office 365 service. However, as we work through building that specific capability, understanding the larger methodology to include the Protection and Detection pillars are key to building confidence in your overall preparedness.
Here are definitions of the pillar terms in relation to your Office 365 services.
- Protection refers to taking advantage of existing capabilities within Office 365 to harden your tenancy against attack. E.g. MFA, password policy, Data Loss Prevention (DLP).
- Detection is the act of processing and analyzing the provided O365 activity information and events to identify and alert on suspected illicit activity within your tenancy
- Investigation defines the forensic data analysis performed on the available Office 365 logs to discover and analyze patterns related to suspected malicious activity.
- Remediation is the collective containment and corrective measures you can take as an Office 365 administrator to limit the activity you have found as well as harden against new activity.
How are my business assets identified within Office 365 ?
The activity data that you can pull from the cloud and any potential actions you would take to improve your cybersecurity stance or respond to an incident revolve around 3 basic entities or classifications of your assets within the Office 365 service: Mail, Document and Account.
- Mail refers to the entire scope of user mailboxes and all messaging contained within (e.g. email, calendar and tasks), the controls within Exchange that affect mail flow rules and filter policies and access identity in relation to those mailboxes (e.g. delegates, shared inboxes).
- Documents are your files stored in OneDrive and\or SharePoint and the identity access control of that information (e.g. documents shared internally/externally, SharePoint site administrator).
- Account defines the scope of user account roles and credential management, tenant level configuration of Active Directory and synchronization between Azure AD and your Office 365 tenancy.
- Administrative actions across the three asset classifications are also audited and logged once enabled.
How does this information increase my preparedness and response?
The process, tooling, and analysis discussed in this post allow you to acquire your Office 365 activity and event data from a central cloud-based data store and analyze on the fly or ingested into the analytical application of your choosing. This provides a scalable platform to glean valuable insight into how protected you are against attacks, your ability to identify or detect suspicious behavior, and how to respond effectively to any future incidents.
The purposes of this series of posts is to assist you to understand a body of knowledge around information security and forensics within the Office 365 service, provide you with guidance and tooling to acquire the activity data logged as part of your subscription as well as how to perform basic investigational and remediation actions within your Office 365 tenancy. Through this series of posts, we will attempt to increase your cybersecurity awareness and capability by providing guidance on the following items.
- Enabling the auditing and logging of activity and event data for your tenancy.
- Acquiring the data from any/all of the Protection Center, PowerShell or API interfaces.
- Analyzing activities in aggregate to rule out known good behavior and identify/track malicious activity.
- Pivoting on an investigational view of activities specific to the suspected malicious activity.
- Executing remediation activities based on the findings from our investigation.
Preparing for cybersecurity response and remediation within Office 365
Review Permissions
The first item we need to address is to ensure you have the proper permissions to access the audit log reporting. You need to be a global administrator for your tenancy or an existing global administrator has to assign you the proper permissions to access the Protection Center, Command shell and API. For more information, review Permissions in Office 365. Please note that Office 365 is migrating the functions and services from the original Compliance Center to the new Protection Center interface (https://protection.office.com).
Enable Activity Logging
The next item on the agenda is to ensure the Exchange Online, SharePoint Online and Azure Active Directory activity and events are being logged. To do this we will need to manually enable auditing, not all activities are audited and logged by default and must be enabled by an account with the proper permissions we outlined in the previous section.
You can enable the basic logging from inside the Protection Center interface. Additional mailbox specific logging can be enabled through PowerShell (including non-owner mailbox access, messaging routing and delivery). Once enabled through the PowerShell interface, this data will be available through all acquisition means discussed in this post.
For additional reference on what information and events are recorded once you enable auditing, please see Audited activities in Office 365.
Protection Center Audit Log
To enable audit logging through the UI, simply navigate to "Search & investigation" and drilling down on the “Start recording user and admin activities” link, see screenshot below. If this link is not shown on your screen, it means logging has already been enabled and you can start using the search interface to acquire data. See the Acquire the data! section below to get started.
Non-owner auditing with Exchange remote PowerShell
To enable the additional mailbox specific events requires a remote PowerShell connection to the Exchange Service. You are required to authenticate yourself against the service, create a remote PowerShell session and enable logging by command line with your specific parameters.
The authentication and remote session pieces of this will be a recurring theme as it establishes a platform to submit commands for discrete queries and actions in your tenancy that we will utilize as part of the investigate and remediate process.
The following script will enable non-owner mailbox access auditing on every mailbox in your tenancy once your session is established.
#Authenticate and issue a credential for the remote PS session
$userCredential = Get-Credential
#Connect to the Exchange remote PowerShell service
$ExoSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $userCredential -Authentication Basic -AllowRedirection
Import-PSSession $ExoSession
#Enable global audit logging for every mailbox in your tenancy
Get-Mailbox -ResultSize Unlimited -Filter {RecipientTypeDetails -eq "UserMailbox"} | Set-Mailbox -AuditEnabled $true -AuditLogAgeLimit 365
#To enable auditing for a single mailbox, replace the content within quotes with either the display name or the email address of the mailbox.
Set-Mailbox -Identity "<display name or address>" -AuditEnabled $true
#Validate logging is enabled.
Get-Mailbox | Select Name, AuditEnabled, AuditLogAgeLimit
For additional information:
- Enabling mailbox logging, please see https://technet.microsoft.com/library/dn879651.aspx
- Connecting to the remote PowerShell service, please see https://technet.microsoft.com/library/jj984289(v=exchg.160).aspx
- Exchange and Shell permissions, please see https://technet.microsoft.com/library/dd638114(v=exchg.150).aspx
Acquire the data!
Please note:
- As stated above in Enable Activity logging, logging must be enabled manually before you can pull any activity data from any interface. If the “Start recording user and admin activity link” is visible on the Audit Log Search page, click to enable auditing.
- It may take up to 12 hours once you have enabled auditing via the user interface before events start showing up in your queries.
- Make sure your start date selected for search within the Protection Center is the day after the day you enabled logging. E.g. if you enabled logging on Feb 3, the earliest day you could start your search by would be Feb 4.
Now that the activities and events we want to look at are being logged, it’s time to grab the data from a number of different interfaces depending on your use case: Protection Center, remote PowerShell, or API. Each of these forms of data acquisition pull from the same data store, the Exchange Audit mailbox, and has its own pros/cons depending on the nature of your request. The Audit mailbox currently retains records for 90 days and this entire scope of data can be queried against with both the Protection Center and remote PowerShell interfaces. The Activity API retains 7 days of data and is designed to be programmatically accessed and ingested into a separate analytic or data store platform.
An example Use Case
An example of a use case implemented across the 3 interfaces shows us the basic function of each and provides context to the query.
A relevant scenario would be: As an Office 365 tenant admin, you need to identify all the SharePoint activities of an individual user over a specific period of time as part of an investigation.
- Navigating to the Protection Center Audit log search feature will provide you the data you require by specifying the time range, choosing the appropriate drop down and filtering results as required.
- The same information is available via PowerShell by utilizing the Search-UnifiedAuditLog cmdlet. You can then view the results in session or output to any number of data formats for ingestion into other analytical platforms.
- The Activity API interface is a bit more complicated to setup, see details in the API section below, however provides a much broader usability as you can programmatically call into the API and facilitate data ingestion into a number of other analytical platforms such as SQL databases, partners with native integrations and/or an industry SIEM.
We dive further into acquiring data from each interface below.
Using the Protection Center
To acquire data from the Protection Center audit log search interface:
- Navigate to https://protection.office.com
- Click through the Search & Investigation option on the left hand navigation menu
- Drill down on Audit Log Search
You can then populate your search by choosing a predefined data set from the Activities drop down and scope your results further by entering users, target files or site in the appropriate search/filter box. You can also take another step by filtering against the individual columns on your results set. Additionally, you can export all results from your discrete query or the entire result data set found for your tenancy to a .csv file for offline analysis.
For additional information about running a search, please see Search the audit log in the Office 365 Protection Center.
Using PowerShell
Establishing an Exchange remote PowerShell session and utilizing the Search-UnifiedAuditLog cmdlet allows you to query the same data store as the Protection Center interface and provides you the ability to fine tune your discrete queries based on the parameters you set on the cmdlet as well as incorporate into scripts for scalable requests.
The following script authenticates against the service, requests a credential and creates a remote PowerShell session. Examples of searching the log for actions of a specific user as well as a SharePoint operations query are added for reference.
Please refer to the TechNet link below for parameter details on the Search-UnifiedAuditLog cmdlet. I also find that utilizing the Windows PowerShell ISE with command add-on (click on View and ensure command add-on is selected) is helpful as it provides range of parameters and context.
#Authenticate and issue a credential for the remote PS session
$userCredential = Get-Credential
#Connect to the Exchange remote PowerShell service
$ExoSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $userCredential -Authentication Basic -AllowRedirection
Import-PSSession $ExoSession
#Search the log for actions of a specific user across Exchange Online, SharePoint Online and Azure AD logs
Search-UnifiedAuditLog -StartDate 10/27/2015 -EndDate 10/29/2015 -UserIds <user@domain.xxx>
#Search the UnifiedAuditLog for SharePoint operations across your tenancy
Search-UnifiedAuditLog -StartDate 12/1/2015 -EndDate 12/10/2015 -RecordType SharePointFileOperation
For additional information:
- Search-UnifiedAuditLog, please see https://technet.microsoft.com/library/mt238501%28v=exchg.160%29.aspx
- remote PowerShell service, please see https://technet.microsoft.com/library/jj984289(v=exchg.160).aspx
Using the Activity API
Calling the Office 365 Management Activity API provides you with information from the same data set as both the Protection Center and Search-UnifiedAuditLog interfaces (Exchange Unified mailbox).
The Activity API is a REST web service that serves as a platform to create new or enhance existing operations and analytics. The interface supports HTTPS and relies on Azure AD and OAuth2 for authentication and authorization. The Management API retains data for 7 days and is designed to be ingested into analytical platforms to provide monitoring, analysis and data visualization important to not only cybersecurity but to a whole range of operational insights within your tenancy.
Our team here at Microsoft has developed some basic PowerShell scripting that allows you to edit a configuration file with your platform details (as of now we support SQL Azure, MySQL and local/Azure file storage), call the API and return data for a number of days or specific date range of data. You can find our tooling in the OfficeDev repository on GitHub: https://github.com/OfficeDev/O365-InvestigationTooling/. Please feel free to provide feedback or contribute. This tooling can only be improved by use, feedback and collaboration with our customers’ on their specific requirements.
For detailed information on building your Azure app, authentication and calling the API, please see Get started with Office 365 Management APIs
Set yourself up to analyze the data
In flight
The data you receive as part of your Protection Center search or remote PowerShell queries can be simply analyzed by reviewing the data as it is delivered to you. The Protection Center interface provides several layers of query and filtering native to the interface as well as the ability to export all results to a .csv file that can be opened and filtered much the same way in Excel. The option to export the partial or full data set from the Protection Center also allows for ingestion into any number of third party or open source platforms we will discuss below.
All the PowerShell queries can be viewed directly in the session and PowerShell provides very powerful tooling to slice, dice and correlate the data to your needs. The data can also be exported to .csv or .json at any time for ingestion into a third party or open source analytical platform.
Third party and Open Source analytical platforms
There are a number of third party and open source analysis tools available that offer differing levels of complexity depending on your security requirements and technical scope. Most options natively ingest the .csv and JSON files produced by the Protection Center and PowerShell, ingesting from the API directly requires additional data structuring especially related to SQL databases.
A number of third party Security Information and Event Management system (SIEM) and analytical applications are working on integration directly with the Office 365 Management APIs. Most of these security focused platforms offer monitoring, detection and alerting as well as other features expected of the industry standard out of the box.
SQL Azure
Using SQL Azure as a data store and analytical platform requires a separate Azure subscription. You can utilize the native SQL tooling or PowerShell for data ingestion into SQL Azure from the .csv exports of both Protection Center and PowerShell. Below are the basic steps to build an Azure SQL store, connect to the database and ingest the Activity data we discussed in this article for analysis. Once you have the SQL Azure database created or already have an existing database, you can leverage our provided PowerShell acquisition and ingestion scripting located on Github to call and ingest your activity data directly into the database.
Basic steps to creating a SQL Azure data store and analytics platform.
- Login to your Azure subscription at https://portal.azure.com
- Ensure you have a storage account setup
- Select + New in the upper left, then Data + Storage, then SQL Database
- Name your new database and make note.
- Select an existing SQL server (and make note of the hostname), or create a new server (making note of the admin account you used to create the DB)
- Select the source, pricing tier, resource group, and associated subscription, then click create.
- Select SQL Servers from the main navigation, select the Server you just created, then click 'Show Firewall Settings'. In the Firewall Settings blade, click 'add client ip' for the host that you will be running the investigations tooling from. Save and wait for confirmation that the firewall rules have been updated.
- Use Visual Studio, or download SQL Server Management Studio Express 2014 (for free) and connect to your new database.
- Create a new SQL database and note the name
- Ensure you have a username and password for an account that can connect to the DB.
MySQL
MySQL is a relatively easy to setup open source relational database management system. You manage the basic administration of your database(s) through the command line interface and download an additional set of tools called MySQL Workbench to easily run and interface with the databases you have created.
Some basic steps to get up and running with a MySQL database:
- Download the free MySQL Installer from their site, https://dev.mysql.com/downloads/windows/installer/
- Open the MySQL command line interface and create your database with the command: "create database o365investigations;"
- Verify the database you created now exists with the command: "show database;"
- Open the MySQL Workbench application and use the wizard to connect to database you just created
- Ensure database is up and running
Once you have your MySQL database created, you can leverage our provided PowerShell acquisition and ingestion script located on Github to call the API and ingest your activity data directly into the database. (https://github.com/OfficeDev/O365-InvestigationTooling/). The script will automatically create the tables require to query your relational database.
Third Party Integrations
Working with partners to develop integrations is key to building this capability for as many Office 365 customers as possible. The following partners have built or are building native integrations that utilize the Office 365 Management API to ingest activity data into their platforms. If you are an existing customer of one or more of these partners, you should be able to start leveraging their feature set today!
Next Steps
The next step of this series of posts will begin to explore the preparedness and response to specific investigation and remediation scenarios, the best practices and cybersecurity rules of thumb that will improve your confidence in the safety of your data in the cloud and increase your operational security rigor to assist in hardening your tenancy against malicious attack.
Blog Resources and Reference
All the TechNet, MSDN and outside source links and references from this post are reiterated in the table below for reference.
| Feature Area | For more information, see... |
| Feature Availability | Office 365 Service Descriptions |
| Permissions | Permissions in Office 365 |
| Auditing reports | Enable mailbox audit logging |
| Audit log search | Searching the audit log |
| Exchange remote PowerShell | Connect to Exchange Online using remote PowerShell |
| Search-UnifiedAuditLog | Search-UnifiedAuditLog |
| O365 Activity API | Get started with Office 365 Management APIs |
| Investigational Tooling | https://github.com/OfficeDev/O365-InvestigationTooling/ |
Additional Office 365 Information and Data Security Resources
Office 365 includes several investigational and remediation features designed to assist you in running a security related investigation outside the scope of the Activity data we discussed in this post. The TechNet article and resource table are included below for your reference.
Using Office 365 tools for security investigations
Additional Office 365 investigational resources not covered in this blog
| Azure AD usage reports | View your access and usage reports |
| Data loss prevention | View reports about DLP policy detections |
| Creating incident reports for DLP policy detections | |
| Create a DLP policy from a template | |
| In-place eDiscovery | Create an in-place eDiscovery search |
| Start or stop an in-place eDiscovery search | |
| Modify an in-place eDiscovery search | |
| Export eDiscovery search results | |
| Create a discovery mailbox | |
| In-place hold | Create or remove an in-place hold |
| Messaging records management | Create a retention policy |
| Add retention tags to or remove retention tags from a retention policy | |
| Apply a retention policy to mailboxes | |
| Place a mailbox on retention hold | |
| Transport rules | Manage transport rules |
| Transport rule actions | |
| Mail protection reports | Mail protection reports using the Excel reporting workbook |
| Trace an email message | Run a message trace and view results |
| Quarantine | Manage quarantined messages |
Comments
- Anonymous
February 15, 2016
JiJi Technologies is a Microsoft Gold Certified Partner, which offers Office 365 Reporting, Auditing and Management solution. I just want to list JiJi Technologies under Third Party Integrations. http://www.jijitechnologies.com/office365-admin.aspxhttp://www.jijitechnologies.com/office365-reports.aspx- Anonymous
February 17, 2016
Added to the list. Thanks so much for the interest!- Anonymous
July 04, 2018
Thanks for the security-focused article."AdminDroid Office 365 Reporting tool" is one of the pioneers of incorporating the O365 Activity API & Search-UnifiedLog cmdlet into their offerings. Please add AdminDroid under the 3rd party integrations list.AdminDroid extends the benefits of the data available from the API by providing a longer data retention period (surpasses the 90 days limit), highly interactive dashboard, periodic scheduling and more.https://admindroid.comLive Demo: http://demo.admindroid.comDownload: https://admindroid.com/download
- Anonymous
- Anonymous
- Anonymous
October 18, 2017
RSA NetWitness integrates with O365 too, Can you please add to the list? Thx.