Why does spam and phishing get through Office 365? And what can be done about it?
Introduction
As a filtering service, Office 365 (Exchange Online Protection, or EOP) is dedicated to providing the best antispam filtering possible, and we take this task seriously:
- We are working hard to keep spam out of your inbox
- We are working hard to ensure we don’t mistakenly mark good email as spam
The question we regularly get from customers is this: Why does spam/phishing/malware get through? Why aren’t you blocking it?
Why spam gets through
Spammers and phishers create malware and send spam because it is profitable. They are always working up new ways to work around spam filters and get messages delivered to user inboxes. Because of the number of unique spammers in the world and the rate at which they create new content, the spam you see in your inbox today is new. It is different than what it was yesterday, or the day before, or the day before that. It looks similar, and may use the same technique, but it is not the same message. It is slightly (or greatly) different and has been designed to evade filters.
Spam campaigns vary in duration. There are some that last many hours, and some that last a few minutes. We have tracked campaigns that send thousands, hundreds of thousands, or even millions of spam messages in under 15 minutes.
When you see spam in your inbox, it is usually because it is a new campaign from a spammer and we do not yet have signatures for it. During this window, a spammer can get some spam through our filter defenses to the inbox. However, our filters catch up and the rest of the campaign is marked as spam.
Image not drawn to scale – we don’t actually miss half the spams
Thus, it is true that some spam gets through. However, a large percentage of it is subsequently caught by one of our anti-spam technologies [1]. End users perceive that we did not catch the spam, but what happens is that the users that are affected are the ones that generate spam complaints, while the ones for whom the filter caught it are unaware that anything was wrong [2].
What you (our customers) can do about it
Office 365 already does several things for spam and phishing filtering [3], but there are a few things that customers can do to help cut down on these types of messages:
1.
Submit spam and phishing samples back to Office 365
This is important!
The reason to submit spam back to us is that it greatly assists in speeding up the discovery of new campaigns as well as the replication of updated signatures. Abuse submissions are combined with multiple other data sources as confirmation signals for faster signature updates. This is true even if we are currently catching the campaign (i.e., user received spam and our signatures subsequently updated, and then the user submits it to us).
[](https://msdntnarchive.z22.web.core.windows.net/media/MSDNBlogsFS/prod.evol.blogs.msdn.com/CommunityServer.Blogs.Components.WeblogFiles/00/00/00/68/90/metablogapi/2783.image_50A7A225.png)
To submit spam to Office 365, please refer to this blog post:
\* Submitting spam to Office 365
[https://blogs.msdn.com/b/tzink/archive/2014/09/12/submitting-spam-back-to-office-365.aspx](../tzink/submitting-spam-back-to-office-365 "https://blogs.msdn.com/b/tzink/archive/2014/09/12/submitting-spam-back-to-office-365.aspx")
Submit malware to Microsoft
If the message is malware and not spam, you can submit it to Microsoft:
* Microsoft Malware Protection Center submission portal
https://www.microsoft.com/security/portal/submission/submit.aspxMicrosoft and Office 365 use these samples to update our anti-malware engines. You can also submit to VirusTotal. Office 365 uses 3 anti-malware engines and all of them are on VirusTotal, who shares samples amongst the other anti-malware companies.
Enable Bulk mail filtering
While neither spam nor phishing, many customers often identify bulk email as spam. The bulk mail feature should be enabled as it can help cut down on the overall level of spam complaints, even if the content is bulk rather than explicitly malicious. For more information, see my previous blog post:
* Different Levels of Bulk Mail Filtering in Office 365
https://blogs.msdn.com/b/tzink/archive/2014/08/25/different-levels-of-bulk-mail-filtering-in-office-365.aspxInvest in User Education
User education is one of the most important aspects of anti-phishing. While technology is one component, users need to be aware of the risks. There are several free resources:
* OnlineGuard.gov’s Antiphishing Page
https://www.onguardonline.gov/articles/0003-phishing* The Anti-Phishing Working Group’s advice to avoid phishing scams
https://apwg.org/resources/overview/avoid-phishing-scamsFor larger organizations, they may want the services of companies that provide anti-phishing education, conducting campaigns to help train users to become more aware of the phishing problem. Two of the ones I am aware of are:
* PhishMe
https://phishme.com/* PhishGuru
https://www.wombatsecurity.com/phishguruA combination of technology plus user education is the best method of preventing falling for phishing scams.
What is Office 365 doing to improve detection of spam and phishing?
There are several different methods that Office 365 is either currently working on or actively investigating to improve our spam, phishing and malware detection capabilities as of Sept 2014. Here is a summary:
1.
Increasing the coverage of URL filtering
EOP currently uses 750,000 URLs in its antispam and antiphishing detection. If a message contains this URL, it is used as a heavy weight in the spam filter.
We are working on increasing this list to well over a million URLs.
**Update: As of December 15, 2014, this is now over 1.7 million URLs\!**
Inbound DKIM verification in IPv4 and IPv6
DKIM is a technology that verifies digital signatures inserted into a message. It is useful for identifying good senders and plays an important role in sorting out good senders from malicious senders.
For more information, see https://tools.ietf.org/html/rfc6376.
Update: As of May 6, 2015, inbound DKIM verification is supported.
Outbound DKIM signing
Office 365 will be giving customers the ability to DKIM-sign all of their outbound email. This will be true of fully hosted customers, hybrid customers or on-premise customers. Customers can either upload their own DKIM keys or let Office 365 generate them.
Update: As of June 2, 2015, outbound DKIM support is under development and should be ready by Q3 2015.
DMARC support
DMARC is a major revolution in spam filtering because it combines both authentication and a feedback loop to help senders improve their authentication practices. But it also was a major step forward in terms of the amount of cross-organization collaboration to come up with a common protocol, and then have everyone implement it.It works by inspecting the From: address, the one that users can inspect, and if it is forged it marks the message as spam or rejects it. Many large brands have implemented DMARC and seen a significant decrease in email spoofing.
DMARC is very useful for detecting phishing and especially spear-phishing.
Update: As of May 6, 2015, inbound DMARC verification is supported. We're still rolling out DMARC reporting.
Faster updates
As you can read above, many of our existing technologies work to catch spam but unfortunately, some of it leaks through before the signatures update. We are currently working on infrastructure to reduce the time start-of-spam-campaign to campaign-detection, and campaign-detection to signature-update.
Update: As of December 15, 2014, the URLs replication has been sped up by 30 minutes!
“New-ness” Inspection
One of the techniques that modern spammers and phishers is to rapidly generate new domains and compromise new machines with IP addresses that have no previous reputation.One technique that Office 365 is investigating is detecting whether or not a given domain or IP is new to the service or new to the Internet. If it is, it can take action by either rejecting the message, temporarily deferring the message or using it as a weight in the spam filter (this is more complicated than graylisting). Good senders will return but many bad senders will not, and that includes spammers and phishers.
Update: As of January 7, 2015, we now do basic IP throttling!
Time-of-Click URL protection
Time-of-Click URL protection involves rewriting the URL of a message to proxy through a service to determine if the destination URL is bad. This occurs when a message has been filtered and deemed non-spam, but after the message is delivered but before the user clicks, the phisher or spammer has uploaded malicious content.In other words, the URL is changed from this:
To this:
https://proxy.example.com/hash/?originalURL=https://www.somedomain.com
The advantage of this feature is that a user is protected even after the message has been filtered and given the wrong categorization (it should be spam instead of good email).
**Update: As of June 2, 2015, time of click URL protection (Safe links) is available for general purchase, see:
- Getting started with Advanced Threat Protection in Office 365, https://www.c7solutions.com/2015/06/getting-started-with-office-365-advanced-threat-protection**- Advanced Threat Protection via Powershell, https://www.c7solutions.com/2015/06/advanced-threat-protection-via-powershellZeroday-protection against malware
Similar to Time-of-Click URL protection, zeroday-protection looks for malware attachments in email that are not caught using standard signature-detection in regular antimalware engines.This is a complex feature that involves multiple moving part components, but suffice to say, it will result in better antimalware detection.
Update: As of May 6, 2015, this type of protection (Safe attachments) is available for general purchase, see:
- Getting started with Advanced Threat Protection in Office 365, https://www.c7solutions.com/2015/06/getting-started-with-office-365-advanced-threat-protection- Advanced Threat Protection via Powershell, https://www.c7solutions.com/2015/06/advanced-threat-protection-via-powershell
Conclusion
We understand the negative experience customers have when they get spam in their inbox. We feel it, too! However, we are working to improve this to ensure that your mailbox stays clean.
[1] The are three types of spam campaigns and their subsequent catch rates:
- 100% catch – these are spam campaigns where we have existing rules and even though the campaign is new, we catch all (or nearly all) of it. This constitutes the largest set of spam campaigns.
- Partial catch – these are spam campaigns where we miss part of it but the filters catch up and catch the rest.
Total miss – spam campaigns where virtually all of it is missed by the filters. This is the smallest set.
Customer complaints are mostly in #2 and #3.
[2] For an overview of how we currently handle spam and phishing, please see the following blog post
* Combating Phishing
https://blogs.msdn.com/b/tzink/archive/2012/08/30/combating-phishing.aspx
[3] To review some of our existing anti-spam documentation:
* How to set up the Office 365 spam filter settings to help block spam
https://support.office.com/en-US/article/How-to-set-up-the-Office-365-spam-filter-settings-to-help-block-spam-da21c0b6-e8f0-4cc8-af2e-5029a9433d59
* Office 365 Email Anti-Spam Protection
https://support.office.com/en-us/article/Office-365-Email-Anti-Spam-Protection-6a601501-a6a8-4559-b2e7-56b59c96a586?ui=en-US&rs=en-US&ad=US
Comments
Anonymous
September 14, 2014
Great article, thanks! What about implementing DANE? tools.ietf.org/.../draft-ietf-dane-srv-02Anonymous
September 15, 2014
DANE is interesting. It doesn't have wide deployment yet but it is something we would consider if there was a strong push towards it.Anonymous
October 09, 2014
appreciate the forward looking enhancements in this article. #5 is of particular interest. any sense of timing on this process improvement?Anonymous
October 13, 2014
@Jmcd: This is on-going and will be done by the end of the calendar year, if not sooner.Anonymous
October 26, 2014
So as i can understand, outbound DKIM is still no supported on Office 365?Anonymous
October 27, 2014
@Andreas: Correct, Office 365 does not yet sign outbound messages with DKIM. It's something we are working on, though.Anonymous
November 13, 2014
Terry, thank you for the informative article. I am particulary interested in #7, the Time-of-Click URL protection, as we have been looking at alternative options to EOP just to gain this feature. Is there a timeline for when this feature is likley to be available?Anonymous
November 15, 2014
@JJ Willow: JJ Willow, I can't reveal the timeline because as far as I know, our Product Marketing Group has not yet committed to a timeline. These next comments are unofficial because we are working through the design and things can change so don't take this as saying "Terry Zink says it will be here on Date X". We believe it will be available in 2015, and I'd like to see it out in the first half of the year.Anonymous
December 07, 2014
use www.antispameurope.com/de this will work properly....Anonymous
May 04, 2015
The comment has been removedAnonymous
June 10, 2015
The comment has been removedAnonymous
July 02, 2015
Don’t get caught in a security trap by solely depending on Office 365 as your security provider. We harden our Email Protection perimeter using the Intel Spambrella service - www.spambrella.com they will provide you with all the reasons to add their service which integrates with Office365. I now read these comment boards knowing I do not have these issues. i spent 3 years fighting Microsoft for resolution which never came.Anonymous
October 28, 2015
The comment has been removedAnonymous
January 20, 2016
I've switched from a Messagelabs provided service, to Office 365. I've gone from zero spam to a deluge. What are Messagelabs doing, that you cannot?Anonymous
February 11, 2016
I just want to say that I like your posting. In fact I am using your site regularly. Your articles are very effective and i am very thankful to you for sharing this site with knowledgeable content .Anonymous
April 16, 2016
Have you tried www.spambrella.com?Anonymous
June 19, 2016
The comment has been removed- Anonymous
June 21, 2016
Office 365 does not use the SORBS IP reputation list. We leverage numerous external and internal IP and URL reputation lists, antimalware vendors, and spam signatures. For some customers, they will want to enabled Advanced Threat Protection which uses sandboxing rather than malware signatures. As always, please report these to us if we are missing them so we can see why they are getting through and make adjustments to our system.
- Anonymous
Anonymous
July 03, 2016
Recently we are informed of massive attacks with zero-day Ransomware. Will Office365 take urgent measures towards such 'special events'?- Anonymous
July 05, 2016
Samir, I am unclear what you mean. Office 365 is always looking for ways to improve its filtering. That includes new technologies, and sometimes we issue special alerts or news bulletins. But usually we are focused on combating threats as we see them and investing in better protection.
- Anonymous
Anonymous
July 06, 2016
Something to think about - particularly with Phishing and Spear Phishing is that there is (and never will be) a technical solution to the problem. The biggest problem is the users themselves falling for the Phish. So you need to train your users to avoid the Phish. Have a look at www.SpearSec.com for example.Anonymous
March 04, 2017
Generally I don't learn article on blogs, however I would like to say that this write-up very forced me to take a look at and do it!Your writing taste has been surprised me. Thanks, very great post.Anonymous
April 18, 2017
Spambrella services doubled up with Office 365 will solve the phish issue.Anonymous
June 14, 2017
always i used to read smaller articles that also clear their motive, and that is also happening with this piece of writing which I am reading at this place.Anonymous
June 16, 2017
I haven't had so much spam since i moved to office 365. Just left an internally hosted exchange with sophos antispam solution and received ZERO spam. i have had 8 spam items in 2 days since moving. heaven knows how much o365 is actually stopping!