Using PAL to Analyze your Performance Counters
Hi all,
I am regularly asked to do a PAL Demo, (Performance Analysis of Logs) so I thought it would be a good idea to write a blog about it. This amazing tool (written by Clint Huffman) will save you time when analyzing the performance counters collected on your servers.
Clint has a blog here: https://blogs.technet.com/clint_huffman
The PAL can be downloaded here: https://www.codeplex.com/PAL (make sure you keep up to date with the newest version!)
1. Install PAL
2. Export or Create a Performance HTM Template
Templates can be exported via the PAL tool under threshold section (SQL Server, BizTalk, Exchange and many others).
When you export a template you will get an initial structure like this: Note: I am using the Processor Counters as an example, you will of course want to add Logical/Physical Disk, Process, all the BizTalk counters etc. etc.
<PARAM NAME="Counter00001.Path" VALUE="\Processor(*)\% Processor Time"/>
<PARAM NAME="Counter00002.Path" VALUE="\Processor(*)\% Privileged Time"/>
<PARAM NAME="Counter00003.Path" VALUE="\Processor(_Total)\% Processor Time"/>
<PARAM NAME="Counter00004.Path" VALUE="\Processor(_Total)\% Privileged Time"/>
<PARAM NAME="Counter00005.Path" VALUE="\Processor(*)\% Interrupt Time"/>
You may however want to change it to something like this:
<PARAM NAME="Counter00001.Path" VALUE="\Processor(*)\*"/>
3. Set up a 'catch' using the template and Perfmon (Start button > run > perfmon) - choose how long you want to collect for (how many hours? what schedule? over day? at night?), choose the interval (bearing in mind if you set 15 secs will a full compliment of counters you are setting a heavy load - watch out not to overload live servers!). You may want to choose a less intensive interval such as every 60 seconds and schedule it to run 24 hours. It is important to set this to be scheduled on all the servers that are involved in your solution. In the case of BizTalk you may want to do the collection on all the BizTalk servers, the SQL Server hosting the BizTalk databases and maybe any other Backend or Frontend servers involved in the process. All collections need to be scheduled to start and stop at the same time. Make sure that you name your collection files (<NAME>.blg) an appropriate name (such as BizTalk01.blg) so that you know which server you are dealing with when you do your analysis later.
4. Once the Performance Counters (Perfmon) have been collected you will have a number of <name>.blg files. You can now start to analyze them.
5. Fire up PAL and go to the counter log screen. Load up the first blg file.
6. go to the Threshold File screen next
a. choose the Threshold File Title - (which type of collection are you doing? BizTalk, SQL etc.)
b. click on the items within the Question Variable Names listbox and set the values appropriately
NumberOfProcessors: open the Task Manager on the server you wish to analysis, go to Performance - the number of boxes under Performance Usage History is the number you fill in
ThreeGBSwitch:
for windows server 2003:
Right-click My Computer and select Properties. The System Properties dialog box will appear.
Click the Advanced tab.
In the Startup and Recovery area, click Settings. The Startup and Recovery dialog box will appear.
In the System startup area, click Edit. This will open the Windows boot.ini file in Notepad.
In the [Operating Systems] section, add the following switches to the end of the startup line that includes the /fastdetect switch: /3GB
Save the changes and close Notepad.
Click OK two times to close the open dialog boxes, and then restart the computer for the change to take effect.
SixtyFourBit: right-click my properties, view the properties
TotalMemory: right-click my properties - can also do a Start > Run > msinfo32. if you save the nfo file you will have all the info you need
KernelDumpOnCDrive: There must be enough free space in the selected location to write the memory dump file. By default, the memory dump file is written to the %SystemRoot%\Memory.dmp file. If there is not enough free space on the %SystemRoot% drive, you can redirect the dump file to another location that has enough free space. To change the dump file path in the Startup and Recovery options on a Windows Server 2008-based computer, follow these steps:
Click Start, right-click Computer, and then click Properties.
Click Advanced system settings on the System page, and then click the Advanced tab.
Click Settings under the Writing debugging information area, and then replace the path with an appropriate value in the Dump file field. In other words, you can change the path from %SystemRoot%\Memory.dmp to point to a local drive that has enough disk space, such as E:\Memory.dmp.
7. Analysis Interval: the tighter the interval the more accurate the results will be however time to create the analysis will increase. A 24 hour catch with 60 second interval * 4 BizTalk Servers * 2 SQL Servers = many hours!
8. OutPut Options:
a. the default output path is [My Documents]\PAL Reports, I personally never use this directory structure for anything because I can never find anything back if I use it. I prefer to create a directory structure similar to C:\Projects\BizTalk\<ProjectName>\Perflogs or something similar.
b. the default file name is: [LogFileName]_PAL_ANALYSIS_[DateTimeStamp]_[GUID].htm
- LogFileName is the name of the blg file that you feed PAL
- DateTimeStamp: self explainitory
- GUID: a crazy long string
Usually you will end up with a crazy (but unique!) file name similar to: "BizTalk01_PAL_ANALYSIS_2009-07-28_14-12-42PM_3B7293C5-653D-45A8-96F5-B4EA6A5E5A0E.htm"
I prefer to just use [LogFileName].htm and save the files in a dated folder. In my experience the long names/paths just result in not being able to copy the files to another folder or CD.
9. Queue: this will just show you the script that is going to be created (a good place to see if you have chosen the correct blg file and corresponding thresholdfile template.
10. Execute: you have 3 choices here
- Execute what is currently in the queue
- Add to queue
- Execute and restart.
*there is also "execute as low priority process", this good for when you still need to do some work and you want PAL to run in the background.
11. Analyze: if everything went according to plan you have analyzed your blg files with PAL. Search for your Output Path (C:\Projects\BizTalk\<ProjectName>\Perflogs or [My Documents]\PAL Reports or whatever you chose) and you should find one folder and one htm file for every blg that you analyzed. The folder simply has all the pics of graphs embedded in the htm file (don't delete this folder or you will loose your cool graphs). Open up the <name>.htm and start reading!
Contained within the PAL output htm file are also a number of links to great articles regarding the Performance Counters that you will be reviewing. These can be a fantastic aid to you when trying to figure what is happening on your servers. The following are just a few of the links to good articles:
Host Throttling Performance Counters: https://msdn2.microsoft.com/en-us/library/aa578302.aspx
How BizTalk Server Implements Host Throttling: https://msdn2.microsoft.com/en-us/library/aa559893.aspx
Threads, DB sessions, and throttling: https://blogs.msdn.com/biztalkperformance/archive/2007/09/28/threads-db-sessions-and-throttling.aspx
Configuration Parameters that Affect Adapter Performance: https://msdn2.microsoft.com/en-us/library/aa561380.aspx
How to Modify the Default Host Throttling Settings: https://msdn2.microsoft.com/en-us/library/aa559628.aspx
Detecting Memory Bottlenecks: https://www.microsoft.com/resources/documentation/windowsnt/4/workstation/reskit/en-us/04memory.mspx?mfr=true