IMF and the Junk E-mail folder in Outlook
I’ve posted several IMF-related blog postings in the past — here, here, and here. In these posts, I’ve focused mainly on the gateway behavior of the IMF. This is because that’s really what the IMF is, of course… just the gateway behavior. But when your mailbox server is running Exchange 2003 or better, there’s a whole different angle to consider: Store behavior.
As I’d mentioned, through the IMF UI in ESM, you can set the Store Action Threshold. This sets a value in the AD that is read by the Information Store on Exchange 2003 (or newer) and controls the server-side Junk E-mail folder behavior. What this means is that if the mail makes it to the Store (ie – is not stopped by IMF at the gateway), it will have an SCL stamped on it. The Exchange 2003 Information Store will detect that there is an SCL value associated with the message. The store will then make a determination on whether or not the message SCL value means it needs to be moved into the mailbox’s Junk E-mail folder.
Important to note that this IMF/SCL related Store Action is totally separate from and unrelated to the Outlook client-side Junk E-mail evaluation. It’s a bit of a confusing interaction, but they really are separate. Even if you never log onto your mailbox with Outlook 2003 in cached mode and set the Junk E-mail options to be enabled, you can still have the Exchange Junk E-mail processing move messages to the Junk E-mail folder. There’s a bit more information on Outlook 2003 client-side Junk E-mail behavior in KB.842510.
Bonus: Outlook doesn’t use the IMF provided SCL values for its client-side (cached-mode only) anti-spam determination. Instead, it does its own Junk E-mail evaluation and determines whether or not to move the mail to the Junk E-mail folder based on the settings within the “Tools->Options” Junk E-mail settings.
- “Low” setting corresponds to approximately a store action threshold of 6 (ie – it will move messages with SCL or 6 or higher to the Junk E-mail folder).
- “High” setting is more aggressive and corresponds to an SCL of between 4 and 5 (ie – it will move messages with SCL greater than somewhere between 4 and 5 to the Junk E-mail folder). Note that there’s not a 1:1 mapping between the four settings in Outlook Junk E-mail behavior and IMF SCLs, so that’s why impacted SCL values are “somewhere between 4 and 5” for Outlook.
Back to the StoreActionThreshold, much of the confusion I see in this area is around what has to be done to get the server-side Junk email processing to work. It’s important to understand that there is actually another requirement that has to be taken into account here. It’s not just the server realizing that the message processed by IMF has a particular SCL and moving it to the Junk E-mail folder… there actually has to be a special rule in place within each mailbox to make this move happen. The rule is made up of information about your junk email settings, safe senders, etc..
Now we’ve established that in order for the Store Action to work properly, you need to have both Exchange 2003 (or better) Information Store that knows to read the StoreActionThreshold value from the AD and ALSO a special rule set in the mailbox to tell the store how to process the Junk E-mail. That’s where most people run into trouble, and why sometimes it seems to work and sometimes it doesn’t. It’s all about how this special rule gets set (or doesn’t)!
- If you fire up Outlook 2003 in cached mode, it will set this special rule in your mailbox automatically.
- If you fire up Outlook 2003 in online mode, it will NOT automatically set this special rule, although you can go into the Tools->Options dialog and change the Junk E-mail configuration to ensure it is set.
- If you fire up Outlook Web Access (OWA) 2003 and go into the Options page, you can check the checkbox to “Filter Junk E-mail”, which creates the special rule.
So, in short, if you’re not having all of your users login to the server with Outlook 2003 in cached mode already, the rule probably isn’t already created and won’t be automatically created just because you implement IMF.
What to do? If you’re not using Outlook 2003 or if you only have a handful of users affected by this behavior, the best option is probably to go into the affected mailboxes (or have the end-user do it) with OWA2003 and set the Filter Junk E-mail checkbox. This will set the special rule in the mailbox, and Junk E-mail folder will begin to function.
But what if you have a BUNCH of mailboxes to set this on? Unfortunately, there’s no built-in way to bulk set this rule on each mailbox directly. But one of my Microsoft colleagues (Jerry Wang) put together a VBScript that will automate the process of spinning through the mailbox in OWA and setting up the special rule. This script was subsequently modified a bit by Xavier Coppin to make it work in more end-user cases. Some credit also belongs with Glen who posted some of these details back in December: https://gsexdev.blogspot.com/2004/12/setting-outlook-2003-junk-email.html. This script might be useful if you’ve a need to set this on a large number of mailboxes. This script comes with no warranty, etc and please review it before you use it, and of course you use it at your own risk!
You feed this script a file containing a list of mailboxes (alias or SMTP address), one per line. If you can’t connect to the mailbox OWA with /exchange//">https://<servername>/exchange/<line-from-the-input-file>/ it won’t work, so construct the input file accordingly and change the strURL value in the script if you need to adjust the URL. The script will prompt for the server, username, password, etc if you don’t provide them according to the usage: cscript JunkEnable.vbs servername domain\user <file name> [Password]
SCRIPT UPDATED Feb 8, 2005
'=======================================================================
' Microsoft provides programming examples for illustration only,
' without warranty either expressed or implied, including, but
' not limited to, the implied warranties of merchantability
' and/or fitness for a particular purpose. This article assumes
' that you are familiar with the programming language being
' demonstrated and the tools used to create and debug procedures.
' Microsoft support professionals can help explain the functionality
' of a particular procedure, but they will not modify these examples
' to provide added functionality or construct procedures to meet your
' specific needs. If you have limited programming experience, you may
' want to contact a Microsoft Certified Partner or the Microsoft fee-based
' consulting line at (800) 936-5200. For more information about Microsoft
' Certified Partners, please visit the following Microsoft Web site:
' https://www.microsoft.com/partner/referral/
'=======================================================================
'==========================================================================
' VBScript Source File
'
' NAME: JunkEnable.vbs
'
' AUTHOR: Xavier Coppin (https://xavblog.net/)
' DATE : 8/Feb/2005
'
' COMMENTS:
' Enable Junk Email filter at mailbox level
'==========================================================================
Option Explicit
'==========================================
Dim ServerName, AdminUserName, Password, oArgs, ArgNumber, TextFileName
Set oArgs = Wscript.Arguments
ArgNumber=CInt(oArgs.Count)
If ArgNumber > 0 Then
If oArgs(0)="/?" Or oArgs(0)="?" Then
Usage()
End If
End If
'WScript.echo "oArgs.count=" & oArgs.count
If ArgNumber < 1 Then
ServerName=InputBox("Can you please enter the Exchange Back-End server name ?","ServerName","ServerName")
Else
ServerName=oArgs(0)
End If
If ArgNumber < 2 Then
AdminUserName=InputBox("Can you please enter an Exchange Administrator account name ?","AdminUserName","DOMAIN\USERNAME")
Else
AdminUserName=oArgs(1)
End If
If ArgNumber < 3 Then
TextFileName=InputBox("Can you please enter the name of the plain text file ?","Text File Name")
Else
TextFileName=oArgs(2)
End If
If ArgNumber < 4 Then
Password=InputBox("Can you please enter the Password of " & AdminUserName,"Password")
Else
Password=oArgs(3)
End If
Dim oFileObject, oLog
Dim strLine,strURL
Set oFileObject = CreateObject("Scripting.FileSystemObject")
Set oLog = oFileObject.OpenTextFile(TextFileName,1)
Do Until oLog.AtEndOfStream
strLine = oLog.ReadLine
strLine = Trim(strLine)
strURL = "https://" & ServerName & "/Exchange/" & strLine
ForceJunkEmailFolder strURL, AdminUserName, Password
Loop
oLog.Close
Set oLog = Nothing
Set oFileObject = Nothing
'==========================================
' Print the usage
'==========================================
Public Sub Usage()
WScript.Echo "Usage:"
WScript.Echo "Enable Filter Junk Email at mailbox level ."
WScript.Echo " cscript JunkEnable.vbs servername domain\user <file name> [Password]"
WScript.Quit 0
End Sub
'==========================================
'Logon to specified mailbox by OWA
'==========================================
Public Sub ForceJunkEmailFolder(strURL,strUserName,strPassword)
On Error Resume Next
Dim XmlHttpStr, ObjXmlHttp
Set ObjXmlHttp = CreateObject("Microsoft.XMLHTTP")
XmlHttpStr = ""
xmlHttpStr = xmlHttpStr & "Cmd=options" & vbLf
xmlHttpStr = xmlHttpStr & "junkemailstate=1" & vbLf
XmlHttpStr = XmlHttpStr & "cmd=savejunkemailrule" & vbLf
ObjXmlHttp.Open "POST", strURL,false,strUserName,strPassword
ObjXmlHttp.SetRequestHeader "Accept-Language","en-us"
ObjXmlHttp.SetRequestHeader "Cache-Control","no-cache"
ObjxmlHttp.setRequestHeader "Content-type:","application/x-www-UTF8-encoded"
ObjxmlHttp.setRequestHeader "Content-Length:", Len(XmlHttpStr)
ObjXmlHttp.SetRequestHeader "Accept", "*.*"
ObjXmlHttp.Send XmlHttpStr
If Err <> 0 Then
WScript.Echo "Filter Junk Email failed at " & strURL & " Failed!"
WScript.Echo "Error number" & Err.Number
WScript.Echo "Description : " & Err.Description
Exit Sub
End If
If ObjXmlHttp.Status <> 200 Then
Wscript.Echo "Filter Junk Email failed at " & strURL & " Failed!"
WScript.Echo "Http status code =" & ObjXmlHttp.status
Elseif ObjXmlHttp.responsetext <> "" Then
WScript.Echo "It seems an error occured during the Send Command. Are you sure you have the correct permissions on this mailbox ?"
WScript.Echo "Response of the server was " & ObjXmlHttp.responsetext
Else
WScript.Echo "Filter Junk Email enabled for mailbox " & strURL
End If
Set ObjXmlHttpStr = nothing
End Sub
Comments
Anonymous
January 01, 2003
Greg presented this Technet evening for me last night.&nbsp; The topic was all about fighting spam -...Anonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
PingBack from http://www.hilpers.com/1175779-imf2-gefilterte-mails-werden-nichtAnonymous
January 01, 2003
I just came across this Blog by Evan and it's does make interesting reading.&nbsp; And the script is...Anonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
The comment has been removedAnonymous
January 31, 2005
The comment has been removedAnonymous
February 06, 2005
Hi Evans, thanks for your excellent blog on this topic.
I have found some interesting behaviour on the store threshold. If the Extended rule is set using Outlook 2003 in cached mode, email with SCL equal or higher than the threshold gets filtered. But if I were to turned on the rule via OWA 2003, only email with a value higher than the threshold gets filtered. I have done the test a few times and I get the same result all the time. Is it possible that you can verify this with someone on the Exchange team?Anonymous
February 07, 2005
Lee -
What you describe wouldn't surprise me at all! The original documentation all discusses treating the Extended rule for StoreActionThreshold as a ">" value, and it's only because Outlook treats it as ">=" rather than as ">" that we have this confusion. It seems quite reasonable to me that Outlook Web Access would treat it as originally documented, although I can see how this could lead to unpredictable behavior. Do you have a case opened with PSS on this issue? If it's causing you trouble in your environment, you may want to pursue that route and see if there's anything that can be done to stabilize this behavior for future versions...
EvanAnonymous
February 07, 2005
Hi Evan,
I think these lines are missing in the ForceJunkEmail sub.
xmlhttp = ""
xmlhttp = xmlhttp & "Cmd=options" & vbLf
xmlhttp = xmlhttp & "junkemailstate=1" & vbLf
Best regards,
Xavier
http://xavblog.netAnonymous
February 08, 2005
Thanks to Xavier for providing some changes to the script and allowing me to post the update. The script now listed in the blog post should now hopefully work for everyone.
EvanAnonymous
February 11, 2005
The comment has been removedAnonymous
February 14, 2005
Not aware of any way to bulk disable the extended rule, although if anyone knows of a way feel free to post it or let me know. You could, of course, turn off the IMF at the gateway to stop the SCL from appearing on the message. This would serve as a sort of "bulk disable" for the extended rule.
Why do you want to disable the rule in bulk? Remember that the more common case is that SOME users have it enabled already (cached mode Outlook 2003) and you realize that other users need to have it added to have consistency.
EvanAnonymous
February 14, 2005
The script worked when I ran it using the Administrator accont and on its corresponding maibox, but it fails with "Http status code=403" if I use on any other mailbox, even with the user's credentials. Help!!! ;)Anonymous
February 14, 2005
Milton -
I haven't run into the 403 errors from the script before, but it wouldn't surprise me at all if it was related to permissions not being in place to log into each user's mailbox. In order to set the options for each mailbox, you will need to have rights to login to the mailbox with the account you specify. Have a look at http://support.microsoft.com/kb/273642 for more information on how to set this up most effectively (it's the same permissions you would need to run Exmerge using this account).
EvanAnonymous
February 15, 2005
Thanks Evan! I tried it. There were a couple of Deny entries at the Organization level for the Administrator account which I removed. Still no go. I should have mentioned is that the Exchange site was set to require https and allow only basic authentication. I tried allowing http access and Windows authentication, but still no go, even after re-setting IIS on the server. Do the Exchange services need to be re-started for any of these changes to take effect? Also, what is the proper syntax for the server name, account name and mailbox name? I've tried the usual but I wonder if the methids being called require something different.Anonymous
February 15, 2005
Hang on - I got it working after disabling basic authentication (I left it on after enabling Windows authentication). According to the script all mailboxes now have the extended rule. But ... no filtering seems to be taking place. I'll try restarting the Exchange services a bit later tonight. How I wish that wasn't necessary to make changes to the IMF settings (it makes the "I" in IMF stand for "Incomplete", "Inconvenient", and "Irritating").Anonymous
February 15, 2005
The comment has been removedAnonymous
February 16, 2005
The comment has been removedAnonymous
February 16, 2005
Milton -
If the SCL is -1, that means it came from a "trusted" (ie - authenticated) source inside your organization. If the SCL is blank, it means the IMF has not processed the message to stamp an SCL on it. There are a number of reasons this might happen. The most obvious thing to check is that your IMF is configured properly on the inbound gateway server's SMTP virtual server, and that it's bound properly to the VS, etc. Less obvious possibilities might range to the "15-character servername" hotfix for IMF. Your best bet if you can't find a ready solution might be to hit the microsoft.public newsgroups, as lots of these sort of IMF issues have been resolved there very successfully.
Evan