Hotmail looking to combat gray mail
Wired yesterday reported about an initiative that Hotmail is working on that combats gray mail – mail that isn’t spam or legitimate mail, but is in-between, a shade of gray. Gray mail is bulk mail and some people want it, while others do not.
Hotmail has come up with the tools to help you get control back. They can help you un-clutter your inbox and give you your time back. Tools such as flagging important emails at the top and the sweep tool to blitz unwanted emails, will help you take control of your inbox. Spend 60 seconds slicing through the clutter with Hotmail's 'Sweep' tool and set rules to automatically schedule filing and deleting emails you don't want in your inbox.
Hotmail has a video of the feature available here: Hotmail Explains the Sweep feature. As someone who has worked in email a long time, I can confirm that gray mail (or greymail) is one of the most difficult types of mail to classify because of its personal component – what is spam to one person is legitimate to another. And while a large proportion of some users may think a message is spam, say 85%, the other 15% wants that mail. Therefore, implementing rules to block the messages globally results in very high rates of false positives. But not blocking them at all results in lots of users complaining about “spam” in the inbox.
This gets complicated as we scale the solution. Some spam filters try to predict or memorize classifications according to what it knows, or thinks, the users want to receive. This works if a spam filter protects several hundred thousand, or even a couple of million, users.
However, if a spam filter has to protect hundreds of millions of users, such as in a large web mail service, there are a lot of resources required to sync all that data and transmit it across data centers. It’s not only a filtering problem, but also an Operations problem – transferring such large data sets strains bandwidth, and processing such huge settings (200 million x 150 custom settings) requires lots of processing power. Just how much effort do you want to put behind a free email service? The law of the rate of diminishing returns applies here.
Finally, greymail is difficult to filter in general because different senders of mail have different reputation classes. There are:
Good mailers who do everything right – they double opt-in users and never spam.
Well-intentioned mailers who do some things right – they may have permission to send but their emails don’t pass SPF checks, they say “To exclude yourself from this message, reply with UNSUBSCRIBE in the subject.” Seriously, what year is this, 2002? Where’s the 1-click unsubscribe?
Mailers who take your contact details from your business card and opt you into their mailing lists. Maybe their services are relevant to you, and maybe they considered you giving them your business card implicit consent to send you mail (‘cause you didn’t read the fine print).
Mailers who can’t be bothered to do anything right and therefore look a lot like spam… but some people might consider them legitimate and if you block it, you get false positive complaints.
Outright snowshoe spammers.
Because of the variations in greymail, it confuses spam filters. Some error on the side of catching more spam (and thus alienating people who want the mail) and some error on the side of allowing more mail (and thus alienating people who don’t want it). The solution is to provide the users tools to quickly and easily deal with their own mail streams.
Kudos to Hotmail for providing tools to allow users to do just that.