SharePoint 2013 Incoming Email - a discussion of some of the requirements
- Cross posted from my post on the SharePoint Team blog, where you'll find blogs from a great bunch of knowledgeable SharePoint support engineers.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
There have been some questions around E-Mail requirements for sending email to a SharePoint list or library, and specifically around the need for the x-sender and x-receiver header fields.
This is not intended to be a comprehensive list of all requirements, but rather an overview of the way we use x-headers and other unique or notable requirements for sending mail to SharePoint lists and libraries. More information about requirements not specifically cited here can be gleaned from the referenced RFC 821 and RFC 822 standards.
We process messages sent to SharePoint lists via email, in three phases.
1. First is the SharePoint Email Engine, which consists of the functionality associated with the SharePoint timer job which fetches the email messages from the Windows SMTP service via the drop folder, and then converts each message into a stream for delivery to the appropriate list in the appropriate SPWeb. This functionality exists in private classes, so there is not a great deal of available information on the internals of this step.
2. The second phase is handled in the SPEmailMessage class. I have linked to the public MSDN information about this step. This phase consists of parsing of the message extracting the header, body and attachment components, and converting these to metadata and attachments to be added to the target list/library is done here.
3. A third phase handles the details of mapping the appropriate metadata and attachments to the library-type specific fields.
X-Headers are typically used in processing SMTP mail from the drop folder. While phase #2 and #3 do not require the X-Headers, phase #1 does require them, and uses them for routing the message.
Since the first phase is not publicly documented on MSDN I have attempted to sum up the requirements for this phase below, with additional, relevant information from phases 2 and 3.
1. Required fields include:
==============================
Sender Header = "x-sender: "
Receiver Header = "x-receiver: "
Mail file Pattern = "*.eml"
Message ID = "message-id"
If there are no Recipients listed in the “x-receiver:” header field, no sender listed in “x-sender:” , or the x-sender header field is not present, we will fail processing the message and return an error with this verbiage in the ULS logs:
-----------------------------------------------------------------------------------------------------------------------------------------
A critical error occurred while processing the incoming e-mail file <email file name> The error was:
Bad senders or recipients
---------------------------------------------------------------------------------------------------------
Additionally, if the message header contains "X-AlertServerType" with a value = "STS" or "X-mailer" with a value = "Microsoft SharePoint Foundation 2010" the message is considered to be originated from SharePoint and will not be delivered to a SharePoint list.
The behavior seen in this first phase is similar to the Exchange replay folder behavior noted here:
https://technet.microsoft.com/en-us/library/bb124230.aspx
The X-Header fields, including x-sender and x-receiver, used by SharePoint are described in more detail here:
https://msdn.microsoft.com/en-us/library/ms526966(v=EXCHG.10).aspx
excerpt:
Using the SMTP Message Envelope Fields
A summary of the use of x-sender and x-receiver from this article is included below:
===========================================================================
The recipientlist field contains the e-mail address of each intended recipient. If the message was submitted over the network through the SMTP protocol, each recipient corresponds to a RCPT TO: protocol command. If the message was submitted through the local SMTP service pickup directory, each recipient corresponds to an x - receiver header at the top of the message file. You can alter this list when executing; for example, you can expand a custom distribution list by substituting the alias address with each member of the list.
The senderemailaddress field contains the address of the message's sender. If the message was submitted over the network using the SMTP, this address corresponds to the transmitted MAIL FROM SMTP command. If the message was submitted using the SMTP service pickup directory, this address corresponds to the x-senderheader at the top of the message file.
Note
The recipientlist and senderemailaddress envelope fields and the various message header fields such as (urn:schemas:mailheader:) To, From, Cc, and Bcc are different attributes of the message. The SMTP does not use the mail header fields to route the message; it routes the message based upon the RCPT TO and MAIL FROM protocol commands if a message is submitted using the SMTP, or the x-receiver and x-sender headers if a message is submitted to the local SMTP pickup directory. The envelope fields are not transmitted or stored with the message and exist only for the message's lifetime in SMTP transport.
===========================================================================
In addition to x-sender and x-receiver, we make use of and populate the messagestatus and arrivaltime x-header fields when picking up the message from the drop folder and delivering it to the appropriate list. These fields help to manage delivery failures, aborts, retries, etc.
2. In the second phase we follow RFC 821 for attributes like “MAIL FROM” (EnvelopeSender), or if not present, we fall back to RFC 822 “FROM” header if MAIL FROM is blank or not present.
The same behavior applies to the RFC 821 “RCPT TO” and RFC 822 “TO” header fields, among others.
In this phase - SPEmailMessage - we use RFC 821 when parsing other message header, body, and attachment components.
The MIME parser used is a specific - and modified to suit SharePoint coding design - version of the Exchange EDGE server role MIME parser.
Of note, a 4K header size limit is imposed by our implementation of the MIME parser.
3. Third phase (SP Email Handler) and additional information:
====================
Attachments - Invalid filename characters = "\\/:*?\"<>|#{}%~&"
SharePoint List types supported for email
NOTE: SharePoint has coded email handlers for the following SPListTemplateType types.
· Announcements
· Events
· DocumentLibrary
· PictureLibrary
· XMLForm
· DiscussionBoard
· Posts