File System Performance Test for USB devices (Compact 2013)
3/26/2014
The File System Performance Test measures the performance of a file system driver (FSD) that uses the Win32 application programming interface. You can use this test to obtain throughput metrics for various file system usage scenarios on a storage device block driver that runs under the FAT, ExFAT, TexFAT, or a custom FSD. The test can also measure the performance of the Object Store file system.
To judge the performance data produced by this test, you need a basis for comparison. For instance, you might use a known theoretical throughput maximum for a specific storage device, or you might compare two data sets obtained by running this test on two different devices, FSDs, or block drivers. The following are examples of useful applications of the File System Performance Test:
* Comparing file system I/O to block driver I/O (the latter can be obtained by running the Storage Device Block Driver Performance Test).
* Comparing file system performance on storage devices from different manufacturers.
* Running a performance test every time there is a known change in any of the underlying components in the file system stack.
This test will generate a number of performance metrics. After running the test, some of these tests can be viewed visually via the GraphTool in the CTK application. Other tests' results can be viewed in the fsperflog.xml log file that is generated. These tests were not graphs because they generate one dimensional or a single datapoint.
Test Name |
In GraphTool |
Description |
|---|---|---|
Throughput Writefile |
Yes |
Measures file write throughput using varying buffer sizes |
Throughput Readfile |
Yes |
Measures file read throughput using varying buffer sizes |
Throughput Writefile Random |
Yes |
Measures the throughput while writing varying buffer sizes of data at random offsets |
Throughput Readfile Random |
Yes |
Measures the throughput while reading varying buffer sizes of data from random offsets |
Throughput Reverse Readfile |
Yes |
Measures the throughput while reading a file in reverse using varying buffer sizes |
Throughput Overwrite |
Yes |
Measures the throughput to overwrite an existing file with varying buffer sizes |
Create Directory Tree |
Yes |
Measures the directory depth that can be created per second |
SetFilePointer and SetEndOfFile |
No |
Measures the time it takes to create a large file with SetEndOfFile |
Seek Speed(read) |
No |
Measures how long it takes to perform a specific number of seeks followed by file reads |
Seek Speed(write) |
No |
Measures how long it takes to perform a specific number of seeks followed by file writes |
Read/Append to Fragmented File |
No |
Time to read and write to a large fragmented file |
Copy Files(from storage to device root) |
No |
Time to copy a set of files from storage to device root |
Copy Files(from device root to storage) |
No |
Time to copy a set of files from device root to storage |
FindFirstFile FindNextFile |
No |
Time to perform FindFileFirst and FindFileNext on 50/100/200 files of the same type |
FindFirstFile FindNextFile Random |
No |
Time to perform random FindFileFirst and FindFileNext on 50/100/200 files of the same type |
Create File CREATE_NEW |
No |
Time to create files with different depths across different directories |
Create File OPEN_EXISTING |
No |
First create files with different depths across different directories and measure the time to CreateFile with the OPEN_EXISTING attribute |
Test Prerequisites
Your device must meet the following requirements before you run this test.
The following table shows the hardware requirement for the File System Performance Test.
Requirement |
Description |
|---|---|
Storage device |
A storage device that is supported by the FSD and storage device block driver; for example a USB hard disk. |
A storage device is not required if the test is intended to measure the performance of the Object Store.
The following table shows the software requirements for the File System Performance Test.
Requirement |
Description |
|---|---|
Tux.exe |
Tux Test Harness, required for executing the test. |
Kato.dll |
Kato logging engine, required for logging test data. |
Fsperflog.dll |
Test library. |
ceperf.dll |
Module containing functions that monitor and log performance. |
perfscenario.dll |
Module containing functions that monitor and log performance. |
Note:When you run the File System Performance Test, the CTK temporarily copies files to the root directory of the target device. Before running the test, verify that there is at least 0.5 megabytes (MB) of free space in the root directory of the device. If there is not sufficient space in the root directory of the device, the test cannot run.
Subtests
The table below lists the subtests included in this test.
SubTest ID |
Description |
|---|---|
2001 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 128 bytes using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 128 byte read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2002 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 256 bytes using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 256 byte read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2003 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 512 bytes using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 512 byte read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2004 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 1KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 1KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2005 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 2KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 2KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2006 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 4KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 4KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2007 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 8KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 8KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2008 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 16KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 16KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2009 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 32KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 32KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2010 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 64KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 64KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2011 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 128KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 128KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2012 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 256KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 256KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2013 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 512KB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 512KB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
2014 |
Throughput Test. Measures the I/O throughput of writing and reading a 2MB file in data blocks of 1MB using the WriteFile and ReadFile functions. The test will also perform random seeks (using the SetFilePointer function) followed by 1MB read and writes. The test fails if the file cannot be created, or if any of the read or write operations fail. |
3001 |
Measures the speed of creating a deep nested directory tree 10 levels deep using the CreateDirectory function. The test fails if any of the directories cannot be created. |
3002 |
Measures the speed of creating a deep nested directory tree 15 levels deep using the CreateDirectory function. The test fails if any of the directories cannot be created. |
3003 |
Measures the speed of creating a deep nested directory tree 20 levels deep using the CreateDirectory function. The test fails if any of the directories cannot be created. |
3004 |
Measures the speed of creating a 500MB file by using SetFilePointer and SetEndOfFile functions. The test fails if the file cannot be created, or if either of the operations fail. |
3005 |
Measures the seek speed by performing 1000 random seeks followed by 1 byte reads using the SetFilePointer and ReadFile functions. The test fails if the file cannot be created, or if any of the seeks or reads fail. |
3006 |
Measures the seek speed by performing 1000 random seeks followed by 1 byte writes using the SetFilePointer and WriteFile functions. The test fails if the file cannot be created, or if any of the seeks or writes fail. |
4001 |
Measures the I/O throughput by continuously reading and appending data to a 100MB file using the SetFilePointer, ReadFile, and WriteFile functions. This test is intended to simulate the behavior of a Digital Video Recorder (DVR) device, which often records and plays back data from the same file. The test fails if the file cannot be created, or if any of the seeks, reads, or writes fail. If the storage device has insufficient space (~300MB), the test will skip. |
4002 |
Measures the speed of copying a folder with 100 files from the storage device to the root directory of the device using the FindFirstFile, FindNextFile, and CopyFile functions. The test fails if the directory or files cannot be created, or if the copy process fails. If this test is run on the Object Store rather than a storage device, then the files will be both created and copied on the Object Store. |
4003 |
Measures the speed of copying a folder with 100 files from the root directory to a storage device using the FindFirstFile, FindNextFile, and CopyFile functions. The test fails if the directory or files cannot be created, or if the copy process fails. If this test is run on the Object Store rather than a storage device, then the files will be both created and copied on the Object Store. |
5001 |
Measures the time to perform FindFileFirst and FindFileNext on same type of 50 files. |
5002 |
Measures the time to perform FindFileFirst and FindFileNext on same type of 100 files. |
5003 |
Measures the time to perform FindFileFirst and FindFileNext on same type of 200 files. |
6001 |
Measures the time to perform FindFileFirst and FindFileNext randomly on same type of 50 files |
6002 |
Measures the time to perform FindFileFirst and FindFileNext randomly on same type of 100 files. |
6003 |
Measures the time to perform FindFileFirst and FindFileNext randomly on same type of 200 files. |
7001 |
Measures the time to CreateFile with the CREATE_NEW attribute up to 10 files across different directories and nested directories. |
7002 |
Creates files with different depths with 10 files across different directories and nested directories. Then measures the time to CreateFile with the OPEN_EXISTING attribute. |
Setting Up the Test
Ensure that a USB storage device is inserted into the SD card slot and properly enumerated. The removable USB storage device must be formatted so that it will be enumerated when plugged in. There are two ways to format the USB storage device:
The storage device can be formatted using a Windows-based desktop machines' built-in formatting application. This assumes the use of one partition.
Running the Test
The File System Performance Test executes the tux -o -d fsperflog.dll command line on a USB storage device on default execution.
The test will generate raw performance data in a log file in the release directory, under fsperflog.log.
You can modify the test by editing the command line. The following table shows the command line parameters for the File System Performance Test.
tux -o -d fsperflog -c "[-p <profile>|-r <path>|-root] [-nb -wt -flush] [-cpu] [-desc <description>]"
Command line parameter |
Description |
|---|---|
-p <profile> |
Specify the storage profile to which to restrict testing, found in the registry under the HKEY_LOCAL_MACHINE\System\StorageManager\Profiles key. |
-r <\path> |
Specify the directory in which to run the test. If you provide this option, the test only runs in the specified directory and ignores the -p option. This flag is useful for a file system driver that does not use storage profiles such as a network file system driver. The path does not require quotation marks; for example, -r \Storage Card |
-root |
Specify that the test should run on the root file system. If you provide this option, the test runs in the root directory and ignores the -p and -r options. |
-nb |
Specify that all the test files should be opened with FILE_FLAG_NO_BUFFERING. This option can be combined with -wt. |
-wt |
Specify that all the test files should be opened with FILE_FLAG_WRITE_THROUGH. This option can be combined with -nb. |
-flush |
Include the calls to the CloseHandle and FlushFileBuffers functions in performance timing calculations. |
-cpu |
Include the CPU usage information in the performance log. |
-desc <string> |
Include a user-specified description string in the performance log. |
-iosize # |
The I/O buffer size for test cases 1001-1005 (default = 512) |
-filesize |
The file size for test cases 1001-1005, 2001-2014 (default = 2MB) |
-rand |
For tests that perform random I/O, uses TUX-provided random seed. |
Verifying the Test
When the test completes running, verify that "PASS" appears in the test log for all sub-tests. A "PASS" test result indicates that the test ran successfully, but the test engineer must manually evaluate the results for each device. Investigate any warning messages that are present in the test log.
After running the test, you can view the results in two ways:
* CEPerf output log - fsperflog.xml
* GraphTool output view
In GraphTool output view, you will be able to see the graphs for tests which generate a series of data. You will have to determine if the performance metrics reported by the test meet the performance required by the device. If not, additional optimizations may be needed. The tests which are graphed are:
* Throughput Writefile
* Thrughput Readfile
* Throughput Writefile Random
* Throughput Readfile Random
* Throughput Reverse Readfile
* Throughput Overwrite
* Create Directory Tree
Troubleshooting the Test
1. Ensure that the USB storage device is formatted with the specific file system that are supported by the test.
2. The device must support USB host functionality.
3. The image must include SYSGEN_USB=1, SYSGEN_USB_STORAGE=1, and SYSGEN_EXFAT=1 in order to have minimal functionality.