News from Tech-Ed 2004 Wednesday - Anti-spam Roadmap
There are three areas that Microsoft is focusing on in their fight against Spam.
- Proof of Identity and Evidence - This will be done with the Coordinated Spam Reduction Initiative (CSRI) using Caller-ID and Computational Puzzles.
- Protect against unwanted email - This can be done with some of the Exchange 2003 anti-spam enhancements, and also being proactive and preventing the spam from reaching the internal network using Edge Services.
- Detect unwanted email - This is done using IMF with SmartScreen technology, 3rd party AV integration, and the Outlook 2003 anti-spam solutions on the client.
More Detail…
Proof: CSRI Roadmap
How will we distinguish legitimate high volume senders from spammers? Caller-ID fights domain spoofing and allows all domains to protect their domain from being used by spammers trying to spoof using their address. As long as they add their outgoing email server in their external DNS they will fine as long as the receiving email servers have Caller-ID.
How will we distinguish legitimate low volume senders from spammers? We could use computational puzzles. This would make a sender that hasn't sent mail to the domain first verify that they are truly a real person by answering a question first (perhaps by reading the text from a picture and replying with it). If they answered correctly they could be trusted. Also, over time, spam filters will become more effective. The end result will be that spam would no longer be as profitable.
Proof: Caller-ID
Senders publish IP addresses for outbound email servers in DNS in an email policy document that is stored as a TXT record. Receivers of email then can determine purported responsible domain of each message by doing a query against the DNS servers for the email policy document of the purported domain and perform domain spoofing test. If it passes, it can let the email through.
So what would you get if you were to try to spoof an email using a microsoft.com domain against an Exchange Edge Server? This nifty error:
550 Originating IP <> is not registered in the email policy document of sender domain microsoft.com
How do you publish a Caller-ID?
Create an _ep TXT record in the following XML format:
<ep.xmlns=https://ms.net/1>
<out>
<m>
<a>IP Address of outgoing servers</a>
</m>
</out>
</ep>
How do you query from this using NSLOOKUP? Microsoft and Hotmail have already published their outgoing server, so lets look at microsoft.com. Here is the output from a command prompt.
C:\>NSLOOKUP
> set q=txt
> _ep.microsoft.com
Non-authoritative answer:
_ep.microsoft.com text =
"<ep xmlns='https://ms.net/1' testing='true'><out><m>"
"<mx/><a>213.199.128.160</a><a>213.199.128.145</a><a>207.46.71.29</a><a>
194.121.59.20</a><a>157.60.216.10</a><a>131.107.3.116</a><a>131.107.3.117</a><a>
131.107.3.100</a>"
"</m></out></ep>"
Cool, yeah?
Protect: Message Taxonomy
We can keep messages from coming to us by looking at the IP address that it comes from. Using Global allow and deny lists in Exchange 2003 you can currently manually configure this. Also you can use DNS Block lists if you like.
We can keep messages from coming to us by blocking the specific sender's email address. We can filter messages sent from particular email addresses or domains. Also, if you know that external email will not com from users in your own domain you can at least prevent internal spoofing.
We can filter the recipients of messages as well. We should block messages to non-existent recipients or specific email recipients like internal Distribution Lists.
Did you know that 54% of mail sent to Microsoft is sent to people that don't exist? Can you imagine what would happen if we sent NDRs back for each of these? Since many of these messages are not legitimate mail we would then get NDRs from the NDRs that we sent even adding more to our email load.
Detect: Anti-Spam (AS) infrastructure
Microsoft's Edge Services will support partner integration. We will expose message properties for authenticated and allowed connections and add per message a Sender Confidence Level rating (SCL) tag. The details for developers on how to do this is already documented in the Exchange SDK. Find more information at the following links:
https://msdn.microsoft.com/exchange
https://msdn.microsoft.com/library/en-us/e2k3/e2k3/ast_anti_spam.asp
Detect: Exchange Intelligent Message Filter (IMF)
This is included in Exchange 2003 with the release of SP1. It is based on SmartScreen Technology developed by Microsoft Research. It is an extension only on Exchange 2003 and will not be on Exchange 2000. It can coexist with 3rd part solutions. Administration is dome from the Exchange System Manager console
Detect: Outlook 2003 enhancements
This allows user specified safe and blocked senders lists. If you receive email that is placed in your Junk E-mail folder because of an SCL that was assigned to the message, you can prevent this from happening in the future by adding the user to your safe recipients list or your contacts. Wether a message is moved to the Junk E-mail folder or not is determined by the Exchange 2003 Mailbox Store based on users lists which are stored there and the SCLs that are put there by IMF or a Client filter. Outlook also blocks all external content by default (Web beacons). This is to prevent spammers from knowing that the email address is valid because it was downloaded from one of their Web servers.
Detect: Anti-Virus (AV) Infrastructure
You have two options as to where to scan in Exchange 2003. You can scan at the Transport level (VSAPI 2.5) or the Store level (VSAPI 2.5, 2.0). Exchange 2003 has backward compatibility with VSAPI 2.0, but added the ability to see additional message properties including the sender email address. Also added was more detailed error reporting.
Future: Exchange Edge Services
The emphasis for this product is on security, extensibility, manageability, and supportability. Core goals are to build a high quality, reliable, secure and scalable SMTP stack that is manageable and easy to deploy. The platform will be very extensible and will focus on anti-spam out of the box.
The primary implementation of the Edge servers will be SMTP. It will stand on its own. This means there will be no infrastructure requirements, Active Directory hard dependencies, domain membership requirements and no Firewall changes required. It is designed to address your important Internet scenarios of handling Internet email traffic, protection from spam e-mail, content filtering, and routing of all messages. The platform will be .NET Framework based allowing you to use Visual Studio to create agents. It can run in secure sandbox environments. It will have rich APIs that allow you to see the queue infrastructure and have full control of message properties.
Where you used to have different server handling different roles you could implement Edge services to handle all of them including: anti-spam, Internet gateways, content filtering, and routing.