Exchange servers send ICMP and UDP packets to clients or Domain Controllers, why?
Hi,
I would like to talk about a few network trace analysis cases where we were requested to find out why certain packets (spefically ICMP and UDP) were sent by Exchange servers. You’ll find below more details about how we found the processes sending those packets:
a) Exchange servers sending UDP packets with random source or destination ports to various clients
In one scenario, our customer’s security team wanted to find out the reason of why the Exchange servers were sending UDP packets to random clients on the network because of security concerns. There was no deterministic pattern regarding source or destination UDP ports. The only consistency was that each UDP packet sent by the Exchange servers had always 8 byte as payload. You can see a sample network trace output below:
Note: Addresses were replaced for privacy purposes even though private IP address space was in use.
No. Time Source Destination Protocol Info
...
105528 2010-01-14 15:20:14.454856 10.1.1.1 172.1.10.14 UDP Source port: 35996 Destination port: mxomss
105530 2010-01-14 15:20:14.454856 10.1.1.1 172.18.10.27 UDP Source port: 35997 Destination port: edtools
105531 2010-01-14 15:20:14.454856 10.1.1.1 172.17.17.95 UDP Source port: 35998 Destination port: fiveacross
105535 2010-01-14 15:20:14.454856 10.1.1.1 172.17.11.51 UDP Source port: 36000 Destination port: kwdb-commn
105540 2010-01-14 15:20:14.454856 10.1.1.1 172.23.98.97 UDP Source port: 36003 Destination port: dicom-tls
105541 2010-01-14 15:20:14.454856 10.1.1.1 172.24.12.8 UDP Source port: 36004 Destination port: dkmessenger
105542 2010-01-14 15:20:14.454856 10.1.1.1 172.28.2.52 UDP Source port: 36005 Destination port: tragic
105545 2010-01-14 15:20:14.454856 10.1.1.1 172.31.5.14 UDP Source port: 36006 Destination port: xds
105546 2010-01-14 15:20:14.454856 10.1.1.1 172.2.10.63 UDP Source port: 36007 Destination port: 4642
105547 2010-01-14 15:20:14.454856 10.1.1.1 172.2.35.68 UDP Source port: 36008 Destination port: foliocorp
105552 2010-01-14 15:20:14.454856 10.1.1.1 172.18.12.55 UDP Source port: 36010 Destination port: saphostctrl
105553 2010-01-14 15:20:14.454856 10.1.1.1 172.48.199.45 UDP Source port: 36011 Destination port: slinkysearch
105554 2010-01-14 15:20:14.454856 10.1.1.1 172.27.133.42 UDP Source port: 36012 Destination port: oracle-oms
105555 2010-01-14 15:20:14.454856 10.1.1.1 172.27.121.40 UDP Source port: 36013 Destination port: proxy-gateway
105558 2010-01-14 15:20:14.454856 10.1.1.1 172.24.7.11 UDP Source port: 36016 Destination port: fcmsys
...
- Source UDP port is increasing and destination UDP port seems random at first sight
- The data part of the UDP datagrams are always 8 bytes. As an example:
Frame 105540 (50 bytes on wire, 50 bytes captured)
Ethernet II, Src: HewlettP_11:11:11 (00:1c:c4:11:11:11), Dst: All-HSRP-routers_15 (00:00:0c:07:ac:15)
Internet Protocol, Src: 10.1.1.1 (10.1.1.1), Dst: 172.23.98.97 (172.23.98.97)
User Datagram Protocol, Src Port: 36003 (36003), Dst Port: dicom-tls (2762)
Data (8 bytes)
=> To better understand which process might be sending that packet, we decided to collect a kernel TCPIP trace on the source Windows 2003 server. You can find more information about methods that could be used to identify the process sending a certain packet, please see my previous post on this
After collecting a network trace and an accompanying kernel TCPIP trace as described in the other post (option 4), we managed to catch the UDP packet that we see in the above network trace (actually the above network trace and the below kernel TCPIP trace were collected together). As an example:
...
UdpIp, Send, 0xFFFFFFFF, 129079416141424158, 0, 0, 2136, 8, 172.023.098.097, 010.001.001.001, 2762, 36003, 0, 0 |
UdpIp, Send, 0xFFFFFFFF, 129079416141424158, 0, 0, 2136, 8, 172.027.153.050, 172.023.021.024, 6004, 36009, 0, 0 |
UdpIp, Send, 0xFFFFFFFF, 129079416141424158, 0, 0, 2136, 8, 172.028.097.111, 172.023.021.024, 2344, 36016, 0, 0 |
UdpIp, Send, 0xFFFFFFFF, 129079416141424158, 0, 0, 2136, 8, 172.027.102.056, 172.023.021.024, 1116, 36022, 0, 0 |
...
- For example, in the red line, 10.1.1.1 (Exchange server) is sending a UDP packet to 172.23.98.97. Packet lenght is 8 bytes and source UDP port is 36003 and destination UDP port is 2762. And process ID that is sending the UDP packet is 2136. Actually in all such UDP packets, process ID is always 2136.
- The above line in Red taken from the kernel trace is the packet #105540 seen in the network trace
=> After checking the “tasklist /SVC” output, we saw that process ID 2136 was store.exe (which is Exchange Information store process):
...
wmiprvse.exe 5176 Console 0 2,168 K
mad.exe 7176 Console 0 45,792 K
AntigenStore.exe 10092 Console 0 200 K
store.exe 2136 Console 0 1,040,592 K
emsmta.exe 12020 Console 0 29,092 K
...
=> After further investigation at Exchange side with the help of an Exchange expert, we found out that this traffic was expected and was used as an E-mail notification mechanism:
https://support.microsoft.com/default.aspx?scid=kb;EN-US;811061 XCCC: Exchange Clients Do Not Receive "New Mail" Notification Messages
The Information Store process (Store.exe) sends a User Datagram Protocol (UDP) packet for new mail notifications. However , because the Store process does not run on an Exchange virtual server but on the cluster node, the UDP packet is sent from the IP address of that node. If you fail over the cluster node, the data and Exchange 2000 Server virtual server configuration are moved to the Store process that is running on the other cluster server node. New mail notifications are sent from the IP address of that second cluster node.
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
b) Exchange servers sending 1 byte pings to DCs
One of our customers reported that their DCs were getting constant ICMP Echo requests from a number of member servers and they wanted to get help in finding the process behind it because of security concerns. After some analysis and testing with the help of an Exchange expert colleague of mine, we found out that those ICMP echo requests were sent by the Exchange server related services. The ICMP echo request has the following characteristics:
=> It’s payload is always 1 byte
=> The payload itself is “3F”
Those ICMP echo requests cease once Exchange server related services are stopped which is another indication. This behavior is partly explained at the following article:
https://support.microsoft.com/kb/270836 Exchange Server static port mappings
Taken from the article:
Note In a perimeter network firewall scenario, there is no Internet Control Message Protocol (ICMP) connectivity between the Exchange server and the domain controllers. By default, Directory Access (DSAccess) uses ICMP to ping each server to which it connects to determine whether the server is available. When there is no ICMP connectivity, Directory Access responds as if every domain controller were unavailable
=> You can also see a sample network trace output collected on an Exchange server:
Hope this helps
Thanks,
Murat
Comments
- Anonymous
October 23, 2012
Good article. The standard MS technotes with TCP/UDP ports from Exchange fails to describe this traffic.