Useful Wireshark Filters for Mail Flow Troubleshooting
There are some problems that you just can’t solve without getting a network capture with tools like Microsoft Network Monitor (superseded by Microsoft Message Analyzer), Microsoft Message Analyzer, or Wireshark. If I had a tag line, it would be, “When in doubt, run Wireshark.” When a problem makes no sense, or you have run out of ideas, you know it’s time for a network capture.
A network trace contains great information, but there is always way more information than we need. In this article I’m going to look at the most common Wireshark filters that I use when I’m troubleshooting mail flow with a network trace. In a previous life I used Wireshark to troubleshoot problems with video streaming, SOAP over HTTP, and server communications, which is why it is my go to tool for network captures.
What sorts of problems require a network capture?
The following are some of the top scenarios where I will want to obtain a network capture.
- SMTP logs show one side is sending the error “500 Unrecognized command”
- Mail server cannot negotiate TLS
- Problems with DNS look ups or timeouts
- I suspect a mail server is sending a message that doesn’t conform to the SMTP RFC
SMTP protocol logs can sometimes be all we need for the above scenarios, but sometimes we will need a network capture to fully know what’s going on.
Obtaining and Using Wireshark
Wireshark is a network protocol analyzer that lets you see what’s happening on the network on a packet level. Wireshark is free to use and can be obtained at https://www.wireshark.org. During the install, you’ll be asked to install WinPcap. This is required for computers that will be taking captures. If you just want to look at capture files, and not actually capture network traffic, WinPcap isn’t required.
Once Wireshark is installed, taking a capture is easy.
Click the options button to verify your configuration.
Ensure that Use promiscuous mode on all interfaces is selected. This will cause Wireshark to also capture traffic that isn’t explicitly destined to, or sent from, the capture machine. Also ensure that no capture filters have been enabled.
I typically don’t’ set a filter on this screen so I can capture as much data as possible. I then rely on filters post capture to parse out unneeded data. It’s better to capture too much, than not enough and be forced to capture again.
Click this button to launch the capture interfaces window.
Check the box beside the interface you would like to capture on, and then click Start.
The capture pane should now start filling with the network traffic.
Replicate the problem, and once completed click the stop capturing button.
Save the capture file.
Filtering in Wireshark
Once a network capture has been obtained we will need to filter out information that isn’t relevant to our investigation. On my non-production mail server, a 5 second capture can result in thousands of captured packets which is overwhelming to look at. With the right filter, I can reduce this to less than 50 packets which will contain a single message hand off.
Once you have entered a filter in Wireshark, ensure you click Apply to see the filtered results.
Filtering by SMTP traffic (port 25)
Assuming traffic is over port 25, there are two ways of doing this. We can type either of the following into the Filter text field.
- SMTP
- tcp.port == 25
I recommend using tcp.port == 25 and not SMTP. The reason being, is that SMTP will only give you SMTP packets. Whereas using tcp.port == 25 will give you the SMTP packets, along with the TCP packets containing SYN, ACK, FIN, and SSL handshakes, which are important to see in a trace.
Using SMTP as the filter I see the following. This message was not sent using TLS so I can see all the communications in the clear.
I see the SMTP communication, but none of the TCP communication that deals with handshakes, re-transmit requests, SSL setup, and other items on the network layer. This may or not be ok. I prefer to see these items so I’ll instead filter by port.
Using tcp.port == 25 as the filter I see the following.
Sometimes there’s a problem with the initial three-way handshake or negotiating TLS, and these are the reasons I like to filter by port 25 and not by SMTP.
Let’s look at a message sent over TLS. Filtering using SMTP shows me the following.
There’s really nothing here because 99% of the communications were sent over TLS which is classified as TCP and not SMTP by Wireshark.
If we instead filter on port 25 we can see additional information like the three-way handshake and the TLS negotiation. We can see the TLS version used, the cipher suite, and the certificate name that was passed. This is especially important to see when troubleshooting a TLS negotiation problem.
Unless you have a particular reason to filter on SMTP, I would recommend using tcp.port == 25 instead.
Filtering by IP address
Let’s say we want to view SMTP traffic to or from a particular IP. Use the following.
ip.addr == 1.1.1.1 and tcp.port == 25
Filtering by DNS requests
Sometimes mail delivery fails because the DNS request performed by the mail server wasn’t successful. To view these results, enter the following in the Filter field.
DNS
In this example, I can quickly see the MX lookup for outlook.com was successful.Click the screenshot for a larger view.
Row 1: Mail server performs a DNS query on outlook.com
Row 2: DNS server returns the MX address for outlook.com
Row 3: Mail server queries the first MX address with the lowest priority. Mx3.hotmail.com
Row 4: DNS server returns the IPs associated with mx3.hotmail.com.
Filter out duplicate rows
Sometimes we need to obtain a capture from a network device like a router. These captures often have duplicate rows present. You’ll see an entry for a packet that enters the router, and then see it again when it leaves the router. For one of my cases, here’s what a capture looked like from a router when I filtered on port 25.
Yuck! I want to filter out the retransmissions and the duplicate ACKs. To do this, I’m going to use the following filter.
tcp.port == 25 and not tcp.analysis.retransmission and not tcp.analysis.duplicate_ack
With this filter in place, the capture is much easier to read.
Filter on a specific conversation
Often you’ll want to filter based on a particular conversation. When a TCP conversation is spun up, the server sending the request will generate a random port. In Wireshark, you can search on the Info column to find a name that would identify the conversation. For example, lets search on the recipient for a message that was not sent using TLS.
In Wireshark, hit CTRL-F, select String, and search for the address.
Once the packet is found, use the bottom pane in Wireshark to determine the ports used.
I can now filter on this TCP Port pair to see this conversation.
There's also another way to isolate a particular conversation, Follow TCP/UDP Stream.
Follow a TCP/UDP Stream
I have one last tip that goes along with the above conversation filter, and is a better way to see only a single conversation. Find a packet that you are interested in, right click, and select Follow TCP Stream.
Wireshark will auto create a filter and apply it (typically it will be something like tcp.stream eq 7). In addition, you’ll see a new window which will show a different view of the conversation. For example, here’s what I see when I follow the TCP stream for a message that was sent not using TLS.
Here we are seeing the raw SMTP commands that the mail server sent for this message. This view can sometimes be easier to view than the packet capture window for a particular conversation. Of course it won’t look this nice for messages sent over TLS.
Summary
The key to analyzing packet captures is to remove as much of the irrelevant data as you can. Although reading the relevant data can be hard, the first step is to filter out what you don’t need.
Do you have tips for reading or filtering packet captures? Leave them in the comments!
Comments
- Anonymous
August 29, 2015
thanks a lot
always interesting - Anonymous
August 21, 2016
good work, very helpful thanks - Anonymous
December 13, 2016
Very useful article.