Filter the security events the OMS Security collects
OMS Security collects all events from Windows Security, App Locker, and Firewall event logs. As OMS is a big data service, it can handle a large amount of data, and customers don’t need to carefully sift through their events to decide which ones to save. But, we have heard from customers that in some cases, they would like to reduce the volume of events that are being collected.
Today, we are enabling a new capability to filter the collected events by selecting sets of events. We wanted to balance the need to reduce the volume while maintaining enough events for investigation, auditing, and threat detection. We also wanted to create a simple mechanism that would allow customers to choose the right filtering policy without maintaining endless lists of event IDs.
With this new capability, OMS Security allows the selection of four sets of events to be collected by the agent. The four sets are:
- All events – For customers who want to make sure all events are collected. This is the way OMS Security worked until now and will continue to be the default.
- Common – This is a set of events that will satisfy most customers and allow them a full audit trial.
- Minimal – A smaller set of events for customers who want to minimize the event volume.
- None – Disable security event collection from security and App Locker logs. For customers who choose this option, their security dashboards will have only Windows Firewall logs and proactive assessments like antimalware, baseline, and update.
These sets were designed to address typical scenarios. Make sure to evaluate which one fits your needs before implementing it.
To choose the set for your workspace, click the Settings button from the main security dashboard:
To determine the events that will belong to each set, we worked with several dozen customers and industry standards. We processed the statistics that they shared with us to learn about the unfiltered frequency of each event and their usage. We used the following guidelines in this process:
- Make sure that the minimal set covers only events that might indicate a successful breach and important events that have a very low volume. For example, this set contains user successful and failed login (event IDs 4624, 4625), but it doesn’t contain logout which is important for auditing but not meaningful for detection and has relatively high volume. Most of the data volume of this set is the login events and process creation event (event ID 4688).
- Provide a full user audit trail in the common set. For example, this set contains both user logins and user logoff (event ID 4634). We also included in the common set auditing actions like security group changes, key domain controller Kerberos operations, and other events that are recommended by industry organizations.
- Events that have very low volume were included in the common set as the main motivation to choose it over all the events is to reduce the volume and not to filter out specific events.
Here is a complete breakdown of the Security and App Locker event IDs for each set:
Other than offering the filtering option, we are always working to improve the parsing and indexing capabilities of the security events. For frequent events, we are making sure that all properties are extracted and indexed. For these events, we removed the EventData property that includes the unparsed XML to reduce their volume.
We are looking for feedback and will fine-tune the event IDs in the different sets. Send us your feedback to: omssecfeedback@microsoft.com. As much as we can, we will try to move events from the bigger sets like all events and common to the smaller sets like minimal and not vice versa.
Comments
- Anonymous
November 16, 2016
Hello Team, could you kindly clarify how this is related to global config option: settings -> data -> windows event logs? Thanks you- Anonymous
August 02, 2017
Its only for Security assessment. The global event collection is for Event logs like Application, System operations manager etc.
- Anonymous
- Anonymous
November 16, 2016
OMS Team, this is fantastic and thank you for listening to your customers on UserVoice. However, there's an immediately glaring omission: A "custom" option to specify your own event IDs. For instance, maybe I want to collect object modifications to OMS but not object access, but our internal policy requires object access to be logged, the difference in logs is massive, but I cannot support that scenario.Why not just add an additional "custom" button that lets me specify the event IDs in a comma separated way just like you do above? All the architecture is there, this is basically just an interface change. - Anonymous
December 20, 2016
dear but how to filter 3 - Network logon events this are in millions - Anonymous
January 19, 2017
Thank you, This can select the Security data volume and cost by each servers/each customers. - Anonymous
January 24, 2017
Helo, it would be great if we could get more details on All, Comon and minimal. including a break down of what is and what is not collected, also an indication on how this might affect storage would be great, i know this will depend on the # of events but some idea based on average usage? - Anonymous
April 09, 2017
Hi oms team. Can I expext the feature "custom filter" event?- Anonymous
May 08, 2017
We need the same -- we need a solution to filter the log Data BEFORE they will send to OMS.
- Anonymous
- Anonymous
May 08, 2017
We need a solution to filter custom log events before they will send to OMS. This is for us critical to filter personal data from log entry's BEFORE they go to the cloud.- Anonymous
May 08, 2017
The best solution would be to be able to define which log entries are sent. For example, only all entries that contain "xyz".
- Anonymous
- Anonymous
June 05, 2017
Hello OMS Team,Hi,I was wondering if MOMS had any way to finetune the frequency of pulling the security baseline per computer agent enrolled in MOMS. The reason I ask is when I try to do remediation for a server sometimes I have to wait maybe half a day or the next day to see the results. If possible I would like to check if I can throttle this to say every 30 minutes or an hour so I can see any actual improvement and remediate as necessary.