Using Columns and Properties

 

You might have had an occasion to add a new column in Network Monitor 3.0. But the list of available choices might be quite daunting. What you may not know, however, is that the list is derived from properties in the NPL script that makes up each parser. This means you can add any kind of information you want as a column.

This blog will attempt to show you some of the more useful columns you can add. But we will also explore the creation of properties in NPL for more advanced analysis.

The Skinny on Columns

It’s probably important to give some background about how columns work in NM3. A common misconception is that the column information is actually the data, but in reality it’s a string describing the data. That means that what’s in the Source Column is not the hex data for 192.168.0.1, but a string that describes it.

This is important because a property is just a variable in NPL that can change based on other factors. For instance the Source column can display the Hardware Address, IPv4 Address, or IPv4 Address based on the frame in question. But in each case, the property is still just a string representation.

Time Related Properties

For starters, let’s display what the default column layout is.

By default, NM3 show “Time Offset” as one of its columns. But it’s often useful to see either the Time of Day or the Time Delta (time from last packet). Well both of these items are available for addition. To add these in, right mouse click any of the column headers and select “Choose Columns…”.

Note that you can also save the current column layout as the default for new sessions, or restore the default column layout. Once you select the menu item, you will see a dialog like the one below.

On the left is a list of column headers you can add, and on the right, your currently enabled column headers. To add a column, select the column on the left and hit add. Or to remove, click on an item on the right and select remove. You can also order the columns using the Move Up/Move Down buttons. It is also possible to move the columns around from the Frame Summary by drag and dropping the columns around, so you don’t need to go to the Choose Frame Summary Columns dialog to do this.

Finding the Column to Add

The first problem you may encounter is that the list is huge, which is why I want to highlight some of the more useful headers you can add without NPL modification. But you still have to find the column, which you could do by just scrolling through the list. But there is a faster way! If you start typing a few of the letters, it should get you to that section more quickly. So, in this example I want to show you how to add the “Time of Day” and “Time Delta” columns. So first click in the Disabled Columns portion of the dialog, and start typing “T I m e” without the spaces, and you see that the items that start with Time appear. The first one you can see is “Time Delta”, so let’s add this to the list by clicking the “Add” button. Then also click on the “Time Of Day” and add it.

Once you click OK from the dialog, you should now see the new columns appear. The “Time Of Day” column will show you the time when the frame was captured. The “Time Delta” displays the time from the last frame. This is updated when a filter is applied, so if frames 1 and 2 are 1 second apart, and frames 2 and 2 are 1 second apart, a filter which only shows frames 1 and 3 will display a time delta of 2 seconds. For example given the following data:

Time Of Day	Time Delta	Frame	Time Offset
15:41:35.331	0.000000	1	0.000000
15:41:36.331	1.000000	2	1.000000
15:41:37.331	1.000000	3	2.000000

If filter that includes frames 2 and 3 are added, the resulting Rime Delta column would look as follows.

Time Of Day	Time Delta	Frame	Time Offset
15:41:35.331	0.000000	1	0.000000
15:41:37.331	2.000000	3	2.000000

Note: If you try to filter on the property for Time Delta in NM3.0, say to find all frames with a delta greater than 1 second, you won’t return any frames. We currently don’t have a way to properly apply this property in a filter. We hope to address this in future version of Network Monitor.

Other Useful Properties

So I wanted to provide a list of other useful properties and their descriptions. This should provide you a reference of some of the more popular column entries.

TCPDescription

It’s often useful to always see the TCP Description rather than the upper protocols. By default, NM3 shows the highest level protocol in the Frame Summary Description field. So by adding the TCPDescription property to the columns, you can always see this information. This is very useful for t-shooting strictly TCP problems.

Other Protocol Description Fields

As with TCP, you may want to see other protocol header rather than other upper level protocols. For instance SMBDescription or HTTPDescription, could be added as columns.

TCPAckNumber, TCPSequenceRange, and TCPFlags

These three properties are also great for t-shooting TCP issues. You may actually find this more useful than the TCP Description as the column information lines up the ACK/SEQ number for easy reference.

SourceNetworkAddress and DestinationNetworkAddress

The way the Source/Destination columns are populated today, you don’t always see the real IPv4 or IPv6 address. Instead we show this based on the following rules. If a higher numbered item exists, we show that item.

1. Alias for the IP address
2. Resolved Name of IP address
3. Real IP address
4. Hardware Address

So in cases where you’d like to see the real IP address and a resolved name exists, turning off the aliases doesn’t show you the real IP address. So you view the real IP address all the time by adding these columns.

SourceHardwareAddress and DestinationHardwareAddress

Like the IP address, the Source and Destination Hardware addresses don’t always show up because of the same rules above. So adding these columns will allow you to always see the Hardware Addresses.

Creating Your Own Properties

On occasion you might find it useful to create your own properties for viewing as a column in NM3. There may be a frame summary data field that you want to show up as a column. You might also want to combine a few data fields and create a column that combines that information. So let’s create a few example properties to show you how this is done.

IPv4 Identification: Adding just a simple property

In this case we’ll create a property for IPv4’s Identification field. This is the field that uniquely tags an IPv4 packet and can be used to associate the same frame from two different captures. We’ll start by opening ipv4.npl. In NM3, you can do this in the parsers tab by searching for the file in the Parser Files in the Object View. Once you find IPv4.npl, double click it to edit. Next, search for the field called Identification, it should look like this:

 UINT16 TotalLength;
UINT16 Identification;

UINT16 FragmentFlags
{
…

Now we add in a property by placing it in square brackets above the data field we want to create a property for.  

 UINT16 TotalLength;
 [IPv4Identification] 
UINT16 Identification;

UINT16 FragmentFlags
{
…

Now save your changed NPL, and reload the parser. From the menu select Tools, Reload parsers. Once the build is complete you can load a trace with IPv4 traffic and add the column you just created.

TCP Port Pair: Adding a derived property with two data fields

In this example, we are going to get a bit fancier. We will create a property that shows both the source and destination port in Hex, along with the friendly port name. Remember that since a property is just a string, we can set it to anything we want. The property does not have to only represent one value.

So in this case we will edit TCP.NPL like we did IPv4.NPL before and look for this section of code.

 UINT16 SrcPort = PortNameTable(this);
[Pair = Port, DestinationPort]
UINT16 DstPort = PortNameTable(this);

We’ll want to add in our property right before the last data field we will reference. In this case there is already a bracketed section above DstPort, but these directive sections can have multiple lines separated by a comma. So we will change this to  

 UINT16 SrcPort = PortNameTable(this);
[
    Pair = Port, DestinationPort,
    TcpPortDesc=FormatString(“%d (%s) -> %d (%s)”, 
    SrcPort, 
    PortNameTable(SrcPort),
    DestPort,
    PortNameTable(DstPort))
]
UINT16 DstPort = PortNameTable(this);

As you see we’ve called our new property TcpPortDesc and we are using FormatString, which is like printf in C, to format the way our property will look. We have 4 parameters for FormatString, the first two are the Source port and friendly name, and the second two are for the destination. We use PortNameTable to convert the number of the port to a descriptive string. PortNameTable is just a table that has already been defined for you in GlobalTables.NPL. Tables are simple structures that given one value, we convert and return a different value.

Once you add in this new column, as described before, you will see a column that displays as follows:

0x50 (HTTP(80)) -> 0xC0E6 (49382)

TCPPayload Info: Adding a property to display TCP payload as ASCII

Often you want to look at the TCP data as ASCII. For FTP conversations, the text reads like a story. But we must also work around a bug in NM3 which limits the text in a field to 0x103 bytes. So what we’ll do is limit the text when we collect it.

Here’s the original section of TCP.NPL we are going to modify.

 //This maybe useful for TCP payload data filter for all TCP based protocols
//A possible filter maybe: Property.TCPPayload.Contains("*****")
//[TCPPayload = AsciiString(frameData, offset, TCPPayloadLength)]
[
DataFieldFrameLength = frameOffset + TCPPayloadLength,

In this case the statement for the entire payload is commented out. You can see that this property may be useful to enable if you want to search TCP Payloads using something like

Property.TCPPayload.Contains(“Anything”)

So we are going to enable this property and add another. Let’s change the code to this:

 //This maybe useful for TCP payload data filter for all TCP based protocols
//A possible filter maybe: Property.TCPPayload.Contains("*****")
[
TCPPayload = AsciiString(FrameData, FrameOffset, TCPPayloadLength),
TCPPayloadDisp = AsciiString(FrameData, FrameOffset, 0x103),
DataFieldFrameLength = frameOffset + TCPPayloadLength,

This adds in two new properties. The first, TCPPayload, will allow us to do searching like described above. The first parameter is the data we are going to search in. When you use FrameData, it means the current frame information. The second parameter is the offset in the buffer, and another global property FrameOffset points to the current location in the frame. The final parameter is the length, and the property TCPPayloadLength holds that value for TCP.

But we can’t use TCPPayload for display because if the length is greater than 0x103 bytes, it won’t get displayed at all. For TCPPayloadDisp we fix this by explicitly setting the length.

It’s All About What You Want to See

Using columns for displaying useful information can save you time and help you t-shoot problems. Perhaps in future version of NM3 we’ll have a way to define standard column setups so you can switch from you TCP T-shooting setup to your SMB t-shooting column setup. Also we do plan to provide some organization to the properties to make it easier to find things in that long list.

Comments

• Anonymous
  January 01, 2003
  This is not some existential trip or search for life's meaning. Rather this refers to a feature protocol

• Anonymous
  January 01, 2003
  This is due to a bug in the NPL.  You can fix this be changing the Window Size in TCP.NPL to: WindowSize = (Conversation.ScaleFactor$[SourceNetworkAddress] >= 0) ? UINT16(FrameData, FrameOffset)  << Conversation.ScaleFactor$[SourceNetworkAddress] : UINT16(FrameData, FrameOffset)

• Anonymous
  January 01, 2003
  Hi Paul, I need to search a capture looking for an ascii string that contains a couple of hex characters.  Is there a way to use "contains" to search for hex data? I tried: property.TCPPayload.contains( 0x1d33303530 ) as a filter.  It didn't generate a syntax error, but also did not find the string. John

• Anonymous
  January 01, 2003
  Property.TCPPayload.Contains(“Anything”) searches for Ascii strings.  How can I search for a Unicode string? I have a trace of some SQL*Server traffic and I would like to search the trace looking for certain strings that are encoded using Unicode by SQL. I was wondering if there was a Uniscodestring that I could use as you use Asciistring. John

• Anonymous
  January 01, 2003
  I did some digging and found that there is a UnicodeString "Custom data type" and I added the following new TCP properties: [TCPPayload = AsciiString(frameData, offset, TCPPayloadLength), TCPPayloadDisp = AsciiString(FrameData, FrameOffset, 0x103), TCPPayloadUnicode = UnicodeString(FrameData, offset, TCPPayloadLength), TCPPayloadUnicodeDisp = UnicodeString(FrameData, offset, 0x103) ] and using the following filter worked! property.TCPPayloadUnicode.contains("string") This is very cool! John

• Anonymous
  January 01, 2003
  Unfortunately the contains plug-in is limited to ASCII or Unicode text.  As soon as non-printable chars are found, the search stops. We do plan to provide a way to search binary data, but I can't say when that will be available. Thanks, Paul

• Anonymous
  January 01, 2003
  I'm wondering if someone encountered the problem that the existing "Windowsize" column sometimes shows just nothing else then blanks....and how to fix this?