DFSR Debug Analysis with Message Analyzer – Part 7, Dealing with Message Analyzer Limitations

This post continues the series that started here.

Default Parser

Up to this point the development of my parser was tested against a single DFSR debug log file. Once I started loading multiple DFSR debug log files, I ran into an issue. I had to choose my parser for every log file I want to parse in the same session –

I discussed this with Paul Long and he was able to provide a terrific solution. I create a file called TextLogConfigMapping.txt with the following contents –

Dfsr*.log = DFSRDebug.config

This simply states that any file name matching Dfsr*.log will use the parser DFSRDebug.config.

I place this file in %LOCALAPPDATA%\Microsoft\MessageAnalyzer and now when I load multiple files, the New Session view defaults to –

Continuous Log Files

After making the change discussed above, I thought I was in a great place to process large numbers of DFSR debug log files together as if they were a single file. Unfortunately there are a few issues –

1. Multi-line messages may span two log files (message begins in log file X and continues in log file X+1)
2. Message Analyzer does not treat consecutive files as a continuous log
3. Every DFSR debug log file begins with a log header and Message Analyzer does not have a method for ignoring messages (every message must be parsed)

The result is that parsing breaks for multi-line messages that span files. To address this, I’ve decided to pre-process my DFSR debug logs at the command line as follows –

copy dfsr*.log combined.log

findstr /v /b /c:"*" combined.log > dfsr.log

This

1. Copies the contents of my DFSR debug log files into a single file called combined.log
2. Extracts all lines not starting with * and puts them in dfsr.log

I’m left with a single file that my parser cleanly handles.

The other issue that arises is that if I’m interested in analysis of just one file that starts in the middle of a multi-line message, the first few lines won’t parse correctly. I’ve chosen to handle this by parsing lines that start with a + with –

/////////////////////////////////////////////////////
// Broken Multi-line continuation
// These may occur at the beginning of a debug
// log if the log rolled over while writing a
// multi-line message
/////////////////////////////////////////////////////

message DfsrMultiLineMessageContinuation with
EntryInfo { Regex = @"(?<MessageText>\+.*)", Priority = 1 },
DisplayInfo { ToText = GetHeaderSummaryText } : LogEntry
{
string Annotation = "Multi-line continuation from a previous debug log";
string MessageText;

    static string GetHeaderSummaryText(any d)
{
var e = d as DfsrMultiLineMessageContinuation;
return e.MessageText;
}
}

Similar to DFSR debug log headers, these lines are injected entirely into a MessageText field and annotated for easy filtering.

Complete Parser

After making the updates in this post and adding a few other specific multi-line message types, v1 of my parser is complete. You may download it here –

DFSRDebug.config

Parser Performance

After combining 323 DFSR debug log files using the method described above, I have a single 3.93 GB log file which Message Analyzer parses in about 20 minutes.

Once parsing is complete, I save the session as a .matp file, reducing file size to 0.99 GB. This file then loads in about 4 minutes with all parsing included – around 8.5 million messages.

Next Up

DFSR Debug Log Charts

Comments

• Anonymous
  January 21, 2016
  The comment has been removed
• Anonymous
  January 24, 2016
  Thanks Jamie :)