Using NMAPI to Access TCP Payload

Artikel
10/07/2009

The TCP Payload often carries data that you want to access directly using the Network Monitor API. Below I will detail how to do this using a simple C++ example and the NMAPI.

Why Not add a TCP.Payload Field?

The TCP Payload can carry all types of payloads depending on the protocol that rides on top of TCP. Most often these represent other protocols, but you might not care about the protocol and instead want to see the payload size or payload data directly. You might think that you could access TCP.Payload to access this data, as this is a valid data field. However, TCP.Payload is only instantiated when no other protocol consumes the data. And in most cases, our parsers are complete enough to attempt to parse the data further. This is a limitation of how NPL works, and means we need to find another way to get the payload data.

Why Not use Property.TCPPayload?

Now there is a property, see this blog for more info on properties, called Property.TCPPayload that you could potentially use. The limitation is that it only works with ASCII or UNICODE data. So for binary information the data does not read properly into the property.

The Solution

The solution is to find the TCP payload depending on the TCP header location and size. We can use Property.TCPPayloadLength to obtain the total length of the payload. And to get the offset into the frame we use the TCP header length (TCP.DataOffset.DataOffset). Finally to get the start of the TCP frame we use the offset of TCP.SrcPort which is the first field in a TCP frame. With these pieces of information, we can use NmGetPartialRawFrame API to grab the raw data from the frame.

So here's the code snippet:

 void
GetFramePayload(HANDLE ParsedFrame, HANDLE FrameParser, HANDLE RawFrame)
{
    ULONG ret;
    UINT32 PayloadLen = 0;
    ULONG retlen;
    NmPropertyValueType PropType;

    UINT8 TCPHeaderSize;
    ULONG TCPSrcOffset, TCPSrcSize;


    // Get Payload Length
    ret = NmGetPropertyValueById(FrameParser, TCPPayloadLengthID, sizeof(PayloadLen), (PBYTE)&PayloadLen, &retlen, &PropType);
    if(ret != ERROR_SUCCESS)
    {
        wprintf(L"Error retrieving TCP Payload Length Property, err=%d\n", ret);
        return;
    }

    if(PayloadLen > 0)
    {
        // Get the Data Offset, used to determine the TCP header size
        ret = NmGetFieldValueNumber8Bit(ParsedFrame, TCPDataOffsetID, &TCPHeaderSize);
        if(ret != ERROR_SUCCESS)
        {
            wprintf(L"Error retrieving TCP Header Length Field, err=%d\n", ret);
            return;
        }

        // Get the Offset of TCP.SrcPort which is the first field in TCP.
        ret = NmGetFieldOffsetAndSize(ParsedFrame, TCPSrcPortID, &TCPSrcOffset, &TCPSrcSize);
        if(ret != ERROR_SUCCESS)
        {
            wprintf(L"Error retrieving TCP SRC Header/Offset, err=%d\n", ret);
            return;
        }

        wprintf(L"Offset: %d, Length: %d, HeaderLen: %d\n", TCPSrcOffset/8, PayloadLen, TCPHeaderSize*4);

        // Allocate a buffer based on the Payload Length Property.
        PBYTE buf = (PBYTE)malloc(PayloadLen);

        // Read in the partial frame.  The Offset is in bits.  TCPHeaderSize is off by a factor of 4.
        ret = NmGetPartialRawFrame(RawFrame, TCPSrcOffset/8 + TCPHeaderSize*4, PayloadLen, buf, &retlen);

        // Do what ever you want with buf now.  I'll assume it's ASCII and print it.
        wprintf(L"%S", buf);
    }
}

And here is the initialization code for each of our frame parser to see how each data field and property was added:

 HANDLE
MyLoadNPL(void)
{
    HANDLE myFrameParser = INVALID_HANDLE_VALUE;
    ULONG ret;

    // Use NULL to load default NPL set.
    ret = NmLoadNplParser(NULL, NmAppendRegisteredNplSets, MyParserBuild, 0, &NplParser);

    if(ret == ERROR_SUCCESS){
        ret = NmCreateFrameParserConfiguration(NplParser, MyParserBuild, 0, &FrameParserConfig);

        if(ret == ERROR_SUCCESS)
        {

            ret = NmAddProperty(FrameParserConfig, L"Property.TCPPayloadLength", &TCPPayloadLengthID);
            if(ret != 0)
            {
                wprintf(L"Failed to add Property.TCPPayloadLength, error 0x%X\n", ret);
            }

            ret = NmAddField(FrameParserConfig, L"TCP.SrcPort", &TCPSrcPortID);
            if(ret != ERROR_SUCCESS)
            {
                wprintf(L"Failed to add field, TCP.SrcPort, error 0x%X\n", ret);
            }

            ret = NmAddField(FrameParserConfig, L"TCP.DataOffset.DataOffset", &TCPDataOffsetID);
            if(ret != ERROR_SUCCESS)
            {
                wprintf(L"Failed to add field, TCP.DataOffset, error 0x%X\n", ret);
            }

            ret = NmCreateFrameParser(FrameParserConfig, &myFrameParser);

            if(ret != ERROR_SUCCESS)
            {
                wprintf(L"Failed to create frame parser, error 0x%X\n", ret);
                NmCloseHandle(FrameParserConfig);
                NmCloseHandle(NplParser);
                return INVALID_HANDLE_VALUE;
            }
        }
        else
        {
            wprintf(L"Unable to load parser config, error 0x%X\n", ret);
            NmCloseHandle(NplParser);
            return INVALID_HANDLE_VALUE;
        }

    }
    else
    {
        wprintf(L"Unable to load NPL\n");
        return INVALID_HANDLE_VALUE;
    }

    return(myFrameParser);
}

By using TCP.SrcPort, we get rid of any dependency of the stack. This will work on IPv4, IPv6 or any tunneled protocols. Also the TCP.PayloadLength is computed by the parsers which again is agnostic to the carrying protocols.

Party on Your Payload

Now that you have your payload in a BYTE buffer, you can do what ever you want with it. For instance, if you wanted to create an expert to show each payload and response as text, you could simply take the frame number that is referenced and use that to determine the conversation key for the TCP conversation, i.e. using a property Conversation.ID.TCP. Then you can use this to filter all other packets in the same trace with the same TCP Conversation ID. This would give you a high level view of text based traffic like HTTP and FTP. Of course there is a little more work to deal with fragmented data, but the API gives you all the tools to accomplish this.

Comments

Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
thanks
Anonymous
January 01, 2003
I'm developing an IDS program that can analyze the payload, if you can help me?
Anonymous
January 01, 2003
Please feel free to post a question in our forums. We can help you out there. http://social.technet.microsoft.com/Forums/en-US/netmon/threads
Anonymous
May 09, 2011
hi i need help... i am a 3rd yr student n doing a project of network monitoring in c#... i am capturing data over a network bt not able to see wat the data is...i need to decrypt it bt dnt knw how....i have all the information abt the header bt no information abt the data...plz plz plz help....its really very urgent...i hav project evaluation...plz help....
Anonymous
February 16, 2014
The comment has been removed

Freigeben über