Storage Block Driver Performance Test for MMC (Compact 2013)
3/26/2014
The Storage Device Block Performance Test is designed to measure the performance of the mass storage block drivers that use the Microsoft Win32 application programming interface. This test can be used to obtain throughput metrics for different block driver usage scenarios for various mass storage devices, since it communicates directly with the device drivers. The device drivers are located below the file system layer and the file allocation table layer. If you have problems with the file system, you can use this test to isolate the problem to a specific layer.
To judge the performance data produced by this test, you need a context as a basis for comparison. For instance, you can use a known theoretical throughput maximum for a specific storage device, or you can compare two data sets obtained by running this test on two different devices, File System Drivers, or block drivers. The following are examples of useful applications of the Storage Device Block Drivers Performance Test:
* Comparing file system I/O to block driver I/O.
* Comparing storage device block drivers performance on storage devices from different manufacturers.
* Running a performance test every time there is a known change in any of the underlying components in the file system stack.
Test Prerequisites
Your device must meet the following requirements before you run this test.
The following table shows the hardware requirement for the Storage Device Block Performance Test.
Requirement |
Description |
|---|---|
A mass storage device |
A mass storage device that works with the block driver being tested. e.g. SD, MMC, and SDHC. |
The following table shows the software requirements for the Storage Device Block Performance Test.
Requirement |
Description |
|---|---|
Tux.exe |
Test harness, required for test execution |
Kato.dll |
Logging engine, required for test execution |
PerfLog.dll |
Logging library that provides functionality for timing and logging the performance data generated by the test dll |
Disktest_perf.dll |
Library containing the test files |
By default, the Storage Device Block Driver Performance Test logs data using the PerfLog.dll library, which allows the user great flexibility in gathering and interpreting the data. The resulting performance data is saved to a file in the \release folder. This data can be exported to .csv format by using the pparse.exe or the PerfToCsv tools, so the data can be viewed in a spreadsheet.
An optional logging method is the BTSPerflog.dll library, which logs performance data in CePerf format. This data can be parsed and displayed graphically via the GraphTool. This requires the following files in addition the software requirement mentioned above.
Requirement |
Description |
|---|---|
btsperflog.dll |
Generate ceperf formatted output |
CePerf.dll |
Module containing functions that monitor and log performance |
perfscenario.dll |
Module containing functions that monitor and log performance |
Subtests
The table below lists the subtests included in this test.
SubTest ID |
Description |
|---|---|
4001 |
Measures the I/O throughput of write and read certain number of sectors (based on the input parameter) at a time to any sector of the disk that can be read back correctly. This test case fails if data is corrupted while reading or writing, or if the boundary of a read or write buffer is overwritten. |
4002 |
Measures the I/O throughput of write and read to multiple scatter-gather (SG) buffers. First, this test case writes data by using a single SG buffer. Then, it reads data by using between 2 and 8 SG buffers of random size and compares the data written to the data read. This test case fails if the driver for the storage device does not support reading multiple SG buffers, if the driver requires individual SG buffers to be sector aligned, or if data is corrupted while reading or writing. |
Setting Up the Test
The Windows Embedded Compact platform under test must have an MMC card inserted and mounted. The removable MMC card must be formatted so that it will be enumerated when plugged in. There are two ways to format the MMC card:
The storage device can be formatted using a Windows-based desktop machines' built-in formatting application. This assumes the use of one partition.
Another recommendation is to turn off cache filtering and/or disk cacheing. These features may affect the raw file system performances.
Running the Test
You can modify the test by editing the command line. By default, this test will test only MMC cards type devices that use the "MMC" profile name.
To specify one or more of the following optional command line parameters for this test, use the -c command line parameter. The following table shows the command line parameters available for the Storage Device Block Driver Performance Test.
Command Line Parameter |
Description |
|---|---|
-zorch |
Required. Enable the functionality of the test. Without this parameter, the test will fail. With this parameter, the test will run and destroy all data on the storage devices. |
-profile profile |
Limits testing to the devices of the specified storage profile. The default is all profiles. |
-disk DSKx: |
Specify the device name of the disk to be tested. If you do not specify this parameter, the test probes for a mass storage disk on device DSK1-9. Specifying this parameter overrides the automatic detection process and forces the test to use the disk that you specify. |
-store |
Specify that the test opens a handle to the disk by using the "OpenStore" function instead of the "CreateFile" function. Opening a handle by using "OpenStore" is not required to test block driver functionality. However, it is useful for testing a storage device that Filesys.exe loads automatically if you cannot open the storage device as a stream device. For example, you can open a handle to a flash driver for ROM by using "OpenStore". |
-perfIterations n |
Specifies the number of times to repeat each timed operation. The default value is 3. |
-numSectors n |
Specifies the number of sectors to read or write during each timed operation. The default value is 3. |
-maxSectors n |
Set the maximum number of sectors per read/write transaction. Specify this parameter for a storage device according to the physical limit on the number of sectors transferred by the storage device at one time. The default value is 16384 sectors. |
-log type |
Specifies how to log the output, as follows: "dbg": log to debug output, "csv": log to .csv file, "perflog": log through perflog.dll. The default value is "perflog". |
-logFile filename |
Specifies the location and name of the .csv file to which the test logs the performance data. To specify a file in the release directory, use \release\filename.csv. If you do not specify this parameter, the test saves the data in the file "disktest_perf.csv" in the root directory. |
-btsperf |
Optional. Generate ceperf/perfscenario formatted output. The performance data file will be saved to CTK's results folder. |
Note:During the test, all information on the storage device is destroyed. The test writes to and reads from sectors on the storage device without preserving the contents of the sectors. Be sure to back up data on all cards and disks that are attached to the tested device before running the test.
Verifying the Test
When the test completes running, verify that "PASS" appears in the test log for all sub-tests. In addition, since this is a performance test, you will have to review the results to ensure that the disk performance meets your expected results.
If you used the perflog.dll library, the logger will generate the performance file name based on the test and system environment information. The file will have a ".LOG" extension in the \release folder.
If you used the optional btsperflog.dll library, the performance data will be saved in the disk_test.xml file in the test results folder created by CTK. You can view the XML file to verify the test results. This test does not currently support graphing via GraphTool.
Troubleshooting the Test
The test starts by displaying basic information about the storage device, containing the name of the device, the handle of the device, the number of sectors on the device, and the number of bytes per sector. The test does not give information about cylinders, heads, sectors, and flags, because this information is not relevant to the Storage Device Block Driver Performance Test.
If the device driver does not return an error, the test then shows how many sectors it writes to and how long, in milliseconds, the operation takes to complete. A message indicates that the test is attempting to read from those sectors. If the device driver does not return an error, the test attempts to verify that the data read is correct.
For more details about the content of the log, see Managing Performance Results Using PerfLog in MSDN. You can choose to write your own parser to convert the raw data or use the default parser, PerfToCsv Tool, provided by Microsoft. Everytime the test uses the perflog.dll library, the logger will append the data to the log file if the file already exists. Therefore, you may want to manually remove or save the previous log file before running the test.