Using PowerShell for Message Analyzer Text Log Parsers
Using PowerShell for Message Analyzer Text Log Parsers
Hi Everyone, Brandon Wilson here with you again to discuss how to use PowerShell to generate parsed data output for log files for Message Analyzer (otherwise known as .matp files). Today we will focus on the text log parsers available as of Message Analyzer 1.3.1, so depending on the version you are running, you may or may not see all of the text log parsers mentioned within this blog. I HIGHLY recommend updating to Message Analyzer 1.3.1 as soon as possible though, as there are a lot of improvements to be had!
Before I get started, it’s important to remember that within Message Analyzer there are a few different feedback mechanisms for you to use (and we want to see your feedback). You can use the feedback link/button in Message Analyzer, reach out in the Message Analyzer forum, or you can leave a comment here, and I will pass it along (I promise). For the Netlogon parser, you can also send an email to MANetlogon@microsoft.comwhere you can submit ideas and any problems or buggy behavior you may run across. I highly recommend reaching out through one of the available methods to provide your suggestions and input!
You can also read up more on Message Analyzer itself at https://blogs.technet.com/MessageAnalyzer
In this walkthrough, we will cover the following:
Saving your output through the GUI
Saving your output through the GUI
Once you open your log file within Message Analyzer directly, you have the ability to save the output as a .matp file. What this does is generate an output file, with the .matp extension, that is already parsed. This provides two major advantages:
1. You don’t have to wait for parsing to complete if you need to review/analyze the file again (or provide it to someone else)
2. In most cases, the file size is reduced vs. the original input file
This is a pretty simple thing to accomplish. Here’s a quick rundown:
1. Open Message Analyzer
2. Open the log file you wish to parse (I will use a Netlogon log in this example)
3. Click the Save button in the top ribbon
4. In the save window that pops up, select either “All Messages” or “Filtered Messages for Analysis Grid view” (depending on if you want to save just a portion you’ve filtered which is showing in the analysis grid or if you want to save all messages). After making that selection, click the “Save As” button in order to save the file as a .matp file.
5. Specify a path and file name for the output file (the default will be the session number; for example, “Session 1.matp”)
And that’s all there is to it…
Available text log parsers
There are a number of text log parsers available for your use. I am not going to cover these in depth within this blog, but I did want to provide you with the options you have available.
So even though its shown above in the screen shot, let’s take a look at what we have here. Most of these are pretty self-explanatory:
1. CheckSymCSV
2. Cluster
3. Dcdiag
4. DPMRegistry
5. EventLogCSV
6. IIS
7. LSP
8. LYNC
9. Netlogon
10. SambaSysLog
11. SCCM
12. SQLServerError
13. SQLServerSetup
14. ULS
15. VMM
16. DefaultSimpleLogFileReaderConfig
Known issues
There are a couple of known issues that have occurred over time, and to be honest, I don’t know that I would actually call them issues (except for the first one).
1. There was a regression in Message Analyzer 1.3 that prevented PowerShell text log parsing from functioning properly. This has been fixed in Message Analyzer 1.3.1.
2. Parsing performance may be a little slower in PowerShell vs. using the GUI. This is in part because PowerShell is not only parsing the data, but it is also saving the .matp output file.
What commands do I need?
For this example, we will use a Netlogon log named DC1-netlogon.log. Fortunately, parsing in PowerShell is pretty straightforward, even for me. In the example, I will show you a way to run the necessary PowerShell commands and pipe it out to a file the old school way, as well as log the performance/runtime of the command itself (just in case you’re interested in such things). Just as a reminder, you can save any of these via the PowerShell ISE, or even via notepad, as a .ps1 file. That file can then be ran either through the PowerShell ISE, or via the PowerShell prompt by using the “.\scriptname.ps1” syntax (no quotes). The same methods below apply to all the text log parsers available through Message Analyzer.
So, what do we ACTUALLY need to do (in the barest form):
1. First, we need to specify a trace session and provide a path for the .matp output
2. Second, we need to provide the path to the log file, as well as the text log parser we want to use
3. Third, we need to actually run the command to begin parsing using the input provided in steps 1 & 2
So, a simplistic example could be this:
$TraceSession01 = New-PefTraceSession -Force -Path B:\temp\DC1-Netlogon.matp -SaveOnStop
Add-PefMessageSource -PEFSession $TraceSession01 -Provider "B:\temp\DC1-Netlogon.log" -LogConfig Netlogon
Start-PefTraceSession -PEFSession $TraceSession01
That’s it; nice and simple right!
Now, let’s say we want to track the performance and write the performance details to an output file (which is handy if you running commands against multiple files, or of course, if you’re just curious).
So, showing you a view of the PowerShell ISE for just one example against a single Netlogon log:
As you can see in the above example, I am doing a basic echo command to tell me which file (in this case, which domain controller) I’m parsing and providing that to a file named Netlogon-tracking.txt located in the B:\temp directory. Next, because I like to know my start (and end) times, I do a simple get-date PowerShell command to output the date and time when the command started (and ended). At that point I kick off the actual meat of the script, except unlike the basic example, I threw a measure-command PowerShell command into the mix in order to evaluate the run times when the script is actually kicked off. After all of that, I simply throw a couple of blank lines in to get ready for any other data I might want to parse as well.
Obviously, all of these extras are optional and aren’t really needed, but I wanted to show you some of the flexibility here.
For reference, this is the type of output that the “measure-command” PowerShell command will get you:
DC1-Netlogon.log...
Friday, July 24, 2015 11:07:41 AM
Days : 0
Hours : 0
Minutes : 0
Seconds : 18
Milliseconds : 660
Ticks : 186609256
TotalDays : 0.000215982935185185
TotalHours : 0.00518359044444444
TotalMinutes : 0.311015426666667
TotalSeconds : 18.6609256
TotalMilliseconds : 18660.9256
Friday, July 24, 2015 11:08:06 AM
Now, let’s say I want to expand on this parsing and review 2 domain controllers and an Exchange server’s Netlogon log. Basically, this is the same as above, except we are calling out more files. Let’s take a look:
Opening your output
Opening the .matp output file is as simple as can be. All you need to do is double click the file to open it, or of course you can right click and select Open (or Open With), or you can open it in Message Analyzer directly using the File menu. Whatever method you choose, when you open the output file, the Message Analyzer GUI will open, and you will be presented with the already parsed output file (as seen in the below screenshot).
Reference links
Message Analyzer v1.3.1 download (highly recommended!)
https://www.microsoft.com/en-us/download/details.aspx?id=44226
Using PowerShell to Automate Tracing
Diving into the Netlogon Parser (v3.5) for Message Analyzer
New Features in the Netlogon Parser (v1.1.4) for Message Analyzer
Introducing the Netlogon Parser (v1.0.1) for Message Analyzer 1.1 (By: Brandon Wilson)
Troubleshooting Basics for the Netlogon Parser (v1.0.1) for Message Analyzer (By: Brandon Wilson)
Quick Reference: Troubleshooting Netlogon Error Codes (By: Brandon Wilson)
Quick Reference: Troubleshooting, Diagnosing, and Tuning MaxConcurrentApi Issues (By: Brandon Wilson)
Message Analyzer Forum
https://social.technet.microsoft.com/Forums/en-US/home?forum=messageanalyzer
Message Analyzer blog site
https://blogs.technet.com/MessageAnalyzer
Memory usage with Message Analyzer
Just to recap; please send us any suggestions or problems you identify through the comments below, the Message Analyzer forum, or using the integrated feedback button in Message Analyzer as seen below (circled in green at the top right)!
That about sums up today’s blog. Can you believe I wrote something this short (I know I hardly can)! Thanks for reading, and as always, I’ll talk to you next time around!
-Brandon Wilson