NPL – The Power Behind the Parsers
I always found it frustrating to use an application, only to find something didn’t work exactly like I expected it to. And in the world of network sniffers, it seems that I always find a problem with a particular protocol or some output that I wish displayed a little differently. With Network Monitor 3.0, you have the ability to modify the source which describes the protocols! This in turn allows you to control what shows up in the frame summary and frame details. And furthermore, allows you to define what is available for columns in the frame summary. Did chills just go up your spine?
The NPL language resembles C++ mixed with IDL’ish (interface description language). For more complicated changes, this may not be for the faint of heart. But as you might be a protocol geek, like me, the leap from understanding how protocols work to how NPL works may not be that big a stretch. And for making simple changes, like summary line modifications, I think it’s in the grasp of most network Admins.
So what can NPL do for me?
While I don’t intend this Blog entry to be a “How to Program in NPL in 5 minutes” I do want to give you an idea of the power in NPL, and perhaps peak your interest. In future Blogs I do plan to highlight some of the details of NPL, but this will just be a taste.
So here is a list of simple modifications you can make by changing the NPLs
· Fix a parser so it displays the network data correctly
· Create a new parser for your proprietary protocol
· Modify the summary description for your specific needs
· Add a new available column with some aggregated data
· Change the port a particular protocol uses
A common problem: “No silly, we do HTTP traffic on port 8888, not 80 or 8080!”
While changing port mappings for protocols could be something revealed in the user interface, we haven’t gotten that far in Network Monitor 3.0 yet. I expect we should address this specific problem on different fronts, i.e. a UI for each protocol, and some way to handle dynamic port allocations. And there are also some heuristics we can use to identify protocols as well. But today, there is a fairly simple way to modify the NPL script for protocols on non-standard ports.
NPL borrows a construct from C called a switch statement. And while it has an overloaded syntax, the basics are the same. A switch statement works by taking a specific data field and then “switching” on different cases depending on the data field’s value. So if the value is 100 you do one thing and if it’s 200 you do something else.
In the context of our problem, making HTTP traffic appear on port 8888, the switch statement is used to decide which protocol to use based on the TCP port number defined. So as you may imagine, TCP.NPL has a switch statement telling it what protocol to use next based on the protocol number defined in the TCP layer.
So here it is, well part of it:
switch(Port)
{
case 20:
case 21:
FTP Ftp;
case 23:
TELNET Telnet;
…
case 80:
case 8080:
Struct
{
Switch
{
Case AsciiString( FrameData, FrameOffset, 4 ) == "COPY":
Case AsciiString( FrameData, FrameOffset, 4 ) == "LOCK":
…
OK, now I know you are probably saying, FTP looks straight forward and so does Telnet, but the stuff below ports 80 and 8080 is starting to look hairy. Well, for our purposes we really don’t care about the info in the case statement. All we are going to do is add another port in the list after 80 and 8080. And here it is…
case 80:
case 8080:
case 8888:
Struct
{
The way the case statement works is that if any of the values (80, 8080, or 8888) match; it will execute the code directly underneath it until the next case. Technically in NPL you only have one statement per set of cases, and in this case the “Struct”.
Once you make the change and save TCP.NPL, just go to the Tools Menu and select Reload Parsers. Once the reload finishes you can reload your trace and now any traffic on port 8888 can be parsed as HTTP.
Can I get a column to display the TCP Sequence Number?
You can probably create a column to display anything you want. You simply define a property in NPL, and it will be available to you in the UI in the column chooser once you rebuild the parsers. A property is simply a variable that is not part of the traced data. So in this case we are going to create a property that stores the TCP sequence number.
[MyNewTCPSeqNumber]
UINT32 SequenceNumber;
The “UINT32 SequenceNumber;” was already there so all we added was the “[MyNewTCPSeqNumber]” line. The way the “[ ]” bracket section works it that it always applies the line right below/next to it. And you can enter multiple statements separated by commas. In this simple case you type a property name and the value of the property is set based on how SequenceNumber is displayed. The display of SequenceNumber is based on the default for UINT32, but I’ll leave the details of that for some other discussion.
So, you could get even fancier by using FormatString.
[MyNewTCPSeqNumber = FormatString(“Seq = %d”, SequenceNumber)]
UINT32 SequenceNumber;
Since the column name is already MyNewSequenceNumber, this is a bit of overkill, but I wanted to show you a simple example of FormatString. This way you can contrive any kind of complex data you want for your column data.
In TCP.NPL we already have defined a property called “TCPSeqNumber” for. But feel free to change this to something else, or add another property by using a comma to separate them.
[
MyNewTCPSeqNumber,
TCPSeqNumber
]
UINT32 SequenceNumber;
Conclusion
This should give you a feeling for the power that NPL can pack. If your interest is peaked and you want to learn more, you can start with the ParserLanguage.doc included with Network Monitor 3.0. I’m sure folks will find interesting ways to extend our parsers, as well as write their own. In fact there’s no reason you couldn’t write a parser for something other than network data, given you could get your data in the Network Monitor file format. The possibilities are endless.
Comments
Anonymous
January 01, 2003
I'm sure you've probably heard this a TON by now... but I'm really frustrated with the difficulty of setting filters. Sorry to make this comparison, but Ethereal makes it so easy to change filters.... Maybe you guys could model Netmon after that a little bit more.Anonymous
January 01, 2003
If you mean applying the filters, we are chaning the UI to make this take one click instead of two. Even today, you can apply the current filter by using Ctrl+Enter (Enter is not an option because we can have multiple lines in our filter). If this is not what you are referring to, please let me know. Thanks, PaulAnonymous
January 01, 2003
thanks