Getting Started with Log Parser Studio - Part 2
In my last post, Getting Started with Log Parser Studio - Part 1, I showed how to get Log Parser Studio along with its minimal prerequisites installed, basic setup as well as running your first query. In this post I'll be taking you on a basic "getting around town" tour to help familiarize you with the LPS Query Library and managing queries. To kick things off let's take a quick look at the library.
Working with the library
Above we see the library that holds all the queries. It's fairly self-explanatory that its a list of all the queries that LPS manages along with a description, date modified, type of query and the query itself (all of these are not visible in the image above). All queries are prefixed with the basic category they reside in. For example a query that queries log files for an IIS website will appear in the following format: IIS: Name Of Query. This makes it easy to visually browse for the query you are looking for.
If you'll remember from my last post I mentioned that the type of query needs to match the type of log being queried. These prefixes directly or sometimes indirectly correlate to those types so if you have IISW3C logs you need queries for, then queries beginning with IIS: are the ones you want. You can also sort the queries by clicking the column header of the field you wish to sort by.
Another advantage to this is searching. Notice the search box at the top right. To quickly narrow down the visible results in the library to list only the log type you need, simply type part or all of any prefix and click the search (>) button. This is free text search of the query name field so you can search for any text contained in any query name no matter where it falls within that string. To clear the results and show all queries again click the X button or press the escape key on your keyboard.
To open any query just double-click it and it will open in it's own tab. You can also right-click a query from within the library for a list of context menu options which are as follows:
Open - Same as double-clicking a query.
Run now - This will open all selected queries then immediately execute them.
Add to batch - This will add the selected queries to the batch manager.
Favorites - Adds selected queries to your favorites list. You can add/remove queries you use the most to your favorites for quick access.
Category - Assign the query(s) to different categories. Note: Due to the ease of searching, static categories may be deprecated in the future.
Delete - Be careful! This choice delete all selected queries. You will get a final warning before doing so. Additionally, this only deletes them from memory. The deletes are not final until you formally close LPS without saving the library.
Hidden feature: CTRL+C. Using this key combination on a selected query will copy the query text to your clipboard. This is so you can quickly take a look at the query in a text editor without having to open it or if you wanted to send the text portion of query as-is to someone via email or any other reason.
Quickly edit query meta-data
For basic edits such as a query's category, name, description, log type etc., its a bit tedious to open it formally, make a simple name change, click save then close the query you aren't even planning on executing. To get around this, select any query in the library then click F2. This will open a basic editor to change those fields. After clicking save, the changes will be propagated back to the library.
Importing and Exporting
Queries can be imported and exported from an entire library, a single query or a group of selected queries. When importing, multiple library XML files can be imported at once, choosing only the queries from each of those files that you want. Once chosen you have the choice to merge these into the library or completely replace the current library. Depending on your workflow this can be very advantageous. You may have certain queries for certain customers, projects or investigations and so on. You could save small groups of queries for certain tasks, export queries to send to others who can import directly into LPS. Or you may simply want only add queries to the default library and export it somewhere so you have a backup. To access importing and exporting go to File > Import or File > Export.
Hidden feature: You can directly open the existing library your default text editor by pressing CTRL+ALT+L. Please be forewarned that this file must be compliant to its format. Translation: make a typo, possibly even a case-sensitive mistake and you won't be able to load this library any longer until you fix the issue. If however, you are a more advanced user and you are aware of the risk involved having direct access to the raw data might be of value. Once it is opened you could also Save-As to another location which is yet another method to back up the library. If you are new to LPS and are wondering how to edit an query, don't do that here, this is accomplished by opening the query in LPS and editing directly in LPS in the query editor window which will be discussed soon.
Backup and Recovery of the Library
The library consists of a single XML file that contains all queries and is stored in the users appdata folder (LPSLibrary.xml). If this file ever becomes lost or corrupted you can recover the default installed library by choosing Help > Recover Library from the main menu bar. However, any queries you have created yourself or existing ones that you have modified will no longer be accessible. If you have custom queries it's a great idea to use the export feature and export your custom queries and/or the entire library to a backup location. If for some reason you need your queries back you can use the import feature to place them back into the library.
Conclusion
The library is the central storage location for your queries. You can execute, modify, import, export, search for and categorize queries. You can backup and restore libraries or parts of libraries and recover the original default library. Queries can be opened for review, then executed or multiple queries can be executed immediately. You can have multiple libraries or groups of queries to suit your working style. The library is typically your home base for managing all your queries, manage it well and it will server you well. Next up working with the query editor.
Comments
Anonymous
January 01, 2003
Regarding the log file types, is it possible to create a custom one without modifying the actual Log Types?Anonymous
January 01, 2003
You can analyze most any type of log with the existing log types. There are about 22 in LPS. For text logs that aren't of a format LPS directly understands you can use Textline log type etc. For what I think you want to do, there should be no problem using LPS, you just may need to create your own queries in LPS and determine which existing log type works best.Anonymous
January 01, 2003
@AnonymousYes, you can create your own Log Types for LP 2.2 but I don't currently have a framework built that allows loading those into LPS dynamically AKA manually adding them to LPS then recompiling.Anonymous
January 01, 2003
Thanks for the answer about log types, so if I wanted to analyze log traffic of the firewalls like cisco ASA, to compare it with that of the IIS, this currently cannot be done with the current log types? I am afraid I didnt properly understand: "manually adding them to LPS then recompiling."Anonymous
January 01, 2003
Hi Anonymous129,
I may have to fix this myself. I've put it on the list of things to check into.Anonymous
January 01, 2003
Thanks for the answer about log types, so if I wanted to analyze log traffic of the firewalls like cisco ASA, to compare it with that of the IIS, this currently cannot be done with the current log types? I am afraid I didnt properly understand: "manually adding them to LPS then recompiling."Anonymous
July 08, 2013
How do you access your Favorite Queries ?Anonymous
July 08, 2013
You'll want to enable the categories option in preferences: "Enable legacy category buttons".Anonymous
September 11, 2013
Great tool Kary, appreciate your work. Great think to add to my workflow. One question - is it possible to specify recursion, like in LogParser with the -recurse option? Thank you!Anonymous
December 11, 2013
Hi Neil, You can set recurse anywhere it was allowed in LP 2.2. You need to click the log format properties button in the query window. It's the icon that looks like a sliver gear just to the right of the drop down where you choose the log type.Anonymous
April 10, 2014
The comment has been removedAnonymous
October 30, 2014
Thanks for these examples.Anonymous
November 21, 2014
how do I review all log items for a single IPAnonymous
March 03, 2015
The comment has been removedAnonymous
November 25, 2015
To review all log items by a single IP Address:
logparser "select cs-uri-stem, count(cs-uri-stem) as requestcount from Logsiise*.log where c-ip = 'Enter IP Address' group by cs-uri-stem order by count(cs-uri-stem) desc"