Separated Cache Solution Evaluation test
Note This content applies to the Windows Logo Kit (WLK). For the latest information using the new Windows Hardware Certification Kit (HCK), see Windows HCK User's Guide on the Windows Hardware Dev Center.
Type: Manual Test
Overview
Separated NVRAM storage solutions provide a number of challenges that are not present in integrated NVRAM storage solutions such as a hybrid hard disk. The purpose of these tests is to verify that, in spite of these additional challenges, both solutions provide the same level of protection and a comparable performance for data that is written to the hard disk.
Details
This test tool consists of two parts:
SCS Evaluation Results Checker: A job that runs in DTM, collects the results from the user, and verifies that all the test cases were run by the developer on the same hardware and driver that is being submitted for logo.
The SCS evaluation results checker job is required for storage adapters that implement a non-volatile cache that is separate from HDDs attached. This is indicated by checking "Implements Non-Volatile cache" when you create a submission in DTM. The job prompts the user for a file when the job is scheduled from Device Console. The user should provide the Results.xml file that is produced from the SCS evaluation test. This job must be scheduled against the storage adapter and driver that were used when running the SCS evaluation test. The Results.xml file must contain passing results for all required tests (see test assertions section).
SCS Evaluation Test : An MSI file that is provided in the Windows Logo Kit (WLK) that is meant to be installed on a developer's machine and used to run the tests and collect results. This does not run as part of DTM. An in depth specification of the SCS evaluation test follows.
Functional Specification
Separated Cache Solution Evaluation test
1 Overview
This document provides the description and motivation for a set of tests that are designed to make sure of the correct functionality of a separated NVRAM storage solution with a focus on data consistency on the hard disk. Separated NVRAM solutions provide a number of challenges that are not present in integrated NVRAM storage solutions such as a hybrid hard disk. The purpose of these tests is to verify that, in spite of these additional challenges, both solutions provide the same level of protection for data that is written to the hard disk.
The document is broken down into sections that contain assumptions, specifications of the test tools and hardware, and detailed descriptions of the tests themselves. We assume that you are familiar with Windows ReadyDrive technology as it is present in Windows Vista, the Non-Volatile Cache Command Proposal for ATA8-ACS e05106r7 specification submitted and accepted by the T13 committee, as well the basic functionality of both Hybrid Hard Disks and separated NVRAM solutions. The Non-Volatile Cache Command Proposal is accessible on the T13 website at T13 Technical Committee.
2 High Level Goals
2.1 Data Loss / Reliability
A primary goal of these tests is to make sure that in a separated NVRAM solution the contents of the disk are always consistent within the operating system. In these scenarios, ensuring consistency primarily relies on the correct ordering of writes and the obeying of flushes issued by the operating system (for example, the FlushFileBuffers API). Detailed descriptions of these two aspects of ensuring data consistency are given later. Note, that these tests would be verified on systems that do not have any NVRAM solution to make sure that there is no hardware or software logic that inherently does not conform to the expected behavior.
A key concern surrounding the use of separated NVRAM solutions is the "recovery process" that occurs in the event of a system crash or power loss. This process is believed to be designed to ensure that in the event of a crash or power loss the data on the disk is at a minimum consistent from the perspective of the operating system. It is acceptable for some rollback to occur provided that rollback does not violate write ordering and flush requirements. While the design and implementation of such a recovery process is not known, it is required that the process be performed correctly. Because this recovery process may rely on data on the disk being in a specific state when the crash or power loss occurred, it is considered invalid to remove the disk and boot it in another system to verify that the contents of the disk are consistent. The recovery process also requires that the disk be allowed to boot up past the BIOS stage prior to a scenario being marked complete. This allows the BIOS bootstrapping code to do its work to attempt to repair or synchronize the disk's contents.
2.2 Performance
It is important that any separated NV cache solution does not adversely impact the performance of the system in key scenarios. These scenarios include steady state performance as well as power transitions such as standby and hibernate. Ensuring a predictable and sufficient level of performance not only improves the quality of the user's experience with Windows Vista but also plays a key part in some reliability scenarios.
The ability to quickly spin the drive down when the system is put into standby or hibernate is critical to ensuring the user's data is not compromised. A common user scenario is to close the lid of a notebook (thus putting it in standby, hibernate, or shutdown) and then picking it up to leave a meeting. If the disk continues to be utilized to write out data during this process it's quite possible the disk arm could be jarred and crashed into the spindle and likely corrupting data.
3 Assumptions
3.1 Hybrid Disk WDK Tests
It is assumed that any device that should be considered for equivalence testing has already passed all of the Hybrid Hard Disk tests in the WDK.
3.2 Baseline System
The "baseline" system for the Separated Cache Solution Evaluation Tests is considered to be a "clean install" for Windows Vista Ultimate on a system with a standard hard disk and the non-volatile cache mode disabled. So, essentially we are compare the same system under two different configurations - non-volatile cache enabled and non-volatile cache disabled. The system is alike in all other aspects.
3.3 Disk Consistency
The consistency of the disk will be ensured by monitoring the IOs issued to the disk using a bus analyzer. The data from the bus analyzer can be compared to the IOs issued by the test in an automated fashion to ensure that the contents of the disk are in a consistent state. As such in this document "consistent on the disk" refers to test detecting that the required IOs have passed across the bus such that they were observed by the bus analyzer. Additional details about the bus analyzer are given in the "Test Tools" section of this document.
3.4 Write Ordering and OS Flush Operations
3.4.1 General Requirements
It is assumed that correct ordering of writes will be maintained at all times. That is given the following series of IOs:
- W1, W2, F1, W3, W4, F2, W5, W6, F3
Where WN are writes issued in the order specified and FN are OS flushes issued in the order specified that only the correct ordering of IOs will be pushed across the bus to the disk. The definition of correct ordering is determined by the configuration of the IO system (write buffering on vs. off, etc). This ordering will be verified using a bus analyzer as described later in this document.
A general rule that applies to the proper ordering of the above IOs requires that after successful completion of any OS flush, FN, all writes issued prior to it are present on the disk. For example, after F2 completes the IOs represented by W3 and W4 must have been observed by the bus analyzer after F1. Additional constraints and requirements are detailed below in configuration specific sub-sections.
3.4.2 Disk Write Buffering Disabled
When the write buffering in the disk's track buffer is disabled proper ordering also requires that prior to the issuing of any given OS flush that the ordering of writes prior to the flush is maintained. In essence with disk write buffering disabled each write must be committed to physical disk spinning media, in order, before the IO is completed to the OS. Given a flush command it is required that data be committed to physical disk spinning media prior to the completion of the flush command.
When configured to have write-buffering in the track buffer disabled the storage sub-system is effectively configured for maximum protection against data loss and inconsistency between the disk and the non-volatile cache. In this configuration the disk's track buffer should be disabled, it is not valid to mask that and leave it enabled. Flushes, FUA, etc. should all be honored according to their defined behavior. While this configuration is not a very common configuration for client systems it is a valid configuration.
3.4.2.1 Disk Write Buffering Disabled - Example
It is assumed that correct ordering of writes will be maintained at all times. That is given the following series of IOs:
- W1, W2, F1, W3, W4, F2, W5, W6, F3
The correct ordering of IOs is as follows:
- W1, W2, F1, W3, W4, F2, W5, W6, F3
It is also valid that in the event of an unexpected system shutdown any partial sequence of the above to be observed provided the ordering is the same.
3.4.3 Disk Write Buffering Enabled
When the write buffering in the disk's track buffer is enabled it is valid for writes to be re-ordered and collapsed provided that ordering around flushes is still maintained. This is typically done to improve performance by optimizing the seek pattern on the disk. As such proper ordering does not require that prior to the issuing of any given OS flush that the ordering of writes prior to that flush operation be maintained.
In the event of a power fail, crash, or other unexpected / dirty shutdown it is valid for the IO sequences to get truncated at any point in time. Once the system has been booted and the recovery operation has been performed the valid sequences as defined earlier must be observed in full on the bus.
This is the common configuration for client systems and as such is given the most attention from both documentation and testing perspectives.
3.4.3.1 Disk Write Buffering Enabled - Reordering Example
It is assumed that flush ordering will be maintained at all times. That is given the following series of IOs:
- W-1.1, W-1.2, F1, W-2.1, W-2.2, F2, W-3.1, W-3.2, F3
It is required that no write W-i.j may be observed on the bus before a write W-(i-1).* has been observed on the bus. That yields the following valid sequences of IOs:
W1, W2, F1, W3, W4, F2, W5, W6, F3
W2, W1, F1, W4, W3, F2, W6, W5, F3
W2, W1, F1, W3, W4, F2, W6, W5, F3
W2, W1, F1, W4, W3, F2, W5, W6, F3
W1, W2, F1, W4, W3, F2, W6, W5, F3
Note that not all valid sequences have been given and that given a sequence of observed IOs it must hold to the general rule in order to be valid.
3.4.3.2 Disk Write Buffering Enabled - Collapsing Example
As stated earlier writes can be not only reordered but collapsed when disk writes occur to the same sector in between two flush operations.
That is given the following set of IOs where (a, b) notation indicates writes to the same sector on the disk:
- W-1.1a, W-1.2, W-1.3b, F1, W-2.1, W-2.2, F2, W-3.1, W-3.2, F3
Yields the following valid sequences of IOs:
W2, W1b, F1, W3, W4, F2, W5, W6, F3
W1b, W2, F1, W3, W4, F2, W5, W6, F3
W1a, W1b, W2, F1, W3, W4, F2, W5, W6, F3
W2, W1a, W1b, F1, W3, W4, F2, W5, W6, F3
Note that not all valid sequences have been given and that given a sequence of observed IOs it must hold to the general rule in order to be valid.
3.4.3.3 Disk Write Buffering Enabled - Unexpected Shutdown Example
In the event of a power fail, crash, or other unexpected / dirty shutdown it is valid for the IO sequences to get truncated at any point in time. Once the system has been booted and the recovery operation has been performed the valid sequences as defined earlier must be observed in full on the bus.
Given the following sequence of IOs:
- W-1.1, W-1.2, F1, W-2.1, W-2.2, F2, W-3.1, W-3.2, F3
It is valid to observe any of the following once the unexpected shutdown has occurred and prior to the recovery having taken place:
W-1.1
W-1.2, W-1.2, F1
W-1.1, W-1.2, F1, W-2.2
Note that not all valid sequences have been given and that given a sequence of observed IOs it must hold to the general rule in order to be valid.
Once the recovery process completes the valid sequences of IOs are the same as in the general case when disk write caching is enabled.
3.4.4 Forced Unit Access (FUA)
Forced Unit Access implies that write through semantics are enforced from the disk write cache's perspective for a given write IO. While a FUA write does not imply a flush of the disk's write cache it is valid for the disk to flush some or part of its disk cache when a FUA write is received.
Given the following sequence of IOs:
- W1, W2_FUA, W3, F1
It is valid to observe any of the following on the bus:
W1, W2_FUA, W3, F1
W2_FUA, W1, W3, F1
W2_FUA, W3, W1, F1
W2_FUA, W1, W3, F1
Note that "W3, W2_FUA, W1, F1" is not a valid pattern. FUA implies that a write has been completed to non-volatile media before any writes after it has been issued. This applies not only to a non-volatile flash solution but also to a hard disk. If these writes are re-ordered in this way and power is lost between the time W3 has completed and W2_FUA has been issued the disk may be in an inconsistent state if removed from the system.
NOTE: FUA Test Cases are NOT part of the LOGO requirements. The test cases included are intended for IHV's who wish to implement FUA in the future.
3.4.5 Write Pattern Optimization
By write pattern optimization we refer to the merging of two or more sector writes into a single write to the disk. Note, that this write may include writing out sectors which are clean. For example, for the case of three contiguous sectors 1,2 and 3 in the NvCache of which sectors 1 and 3 have been modified one can have a single write which comprises these three sectors even though sector 2 is clean.
A caveat for this optimization is that sectors should be evicted from the NvCache only after they have been committed to disk successfully. This is because a power failure when the write is in progress might leave the disk in an inconsistent state and possibly even corrupt data which was considered up-to-date on the disk (sector 2 in the example above).
Note that such an optimization may not be valid if the writes are a mix of FUA writes and non-FUA writes. Any ordering constraints must be obeyed ahead of any optimizations such as this. The concern also applies to the case when write-buffering is disabled and any merging optimizations can be applied only while adhering to the ordering constraints.
3.4.6 Other Write / Flush Optimizations
3.4.6.1 Bonus Flushes
It is valid for the IO subsystem to issue additional flushes to the disk provided the write ordering rules and tests outlined in this document are not violated.
It is important to ensure the performance requirements and tests in this document are also not violated.
3.4.6.2 Write Buffer Optimizations in the Track Buffer
There are some concerns which suggest disabling the write-buffer by default may not be desirable. Write-buffering is typically enabled by default on hard disks. Moreover, an end-user can toggle this feature i.e., disable or enable write-buffering as desired. In some sense, this default then indicates the user's choice. Changing this default then reflects a departure from the conviction that user settings are being adhered to. If one argues that, with write-buffering turned off, flushes are still sent to the non-volatile cache instead of disk media, is indeed in the user's best interest as it delivers improved performance, then extreme care needs to be taken in the instrumentation. In particular, the actual user setting should be remembered and appropriately reset as and when the system transitions to non-volatile cache disabled from non-volatile cache enabled[1]. If the user had write-buffering enabled originally and the user-setting is not reset the user will see a significant performance impact as all writes will be write-through (one of the primary reasons why write-buffering is enabled by default).
Finally, note that for the case when write-buffering is disabled on the disk then all flush commands as well as write-through flags are removed by the Windows operating system before any request is sent out over the wire. So, no additional logic needs to be implemented to strip the flushes. Please see the following document for details.
Disk Subsystem Performance Analysis for Windows
3.5 Performance
As described earlier performance of a separated cache solution is a concern and ensuring that it provides a level of performance that is consistent and performant is critical.
3.5.1 Hybrid Disk Throughput / Latency Requirements
Hybrid Hard Disks are required to provide the following level of performance in order to be used by the system as an NV cache enabled solution:
16 MB/s sustained throughput for 64 KB sequential reads when the platter is spun down to a file pinned in the NV cache.
8 MB/s sustained throughput for 64 KB sequential writes when the platter is spun down.
4 MB/s sustained throughput for 4 KB random reads and writes when the platter is spun down to a file pinned in the NV cache.
Less than 1 ms latency for a given 4 KB random read when the platter is spun down to a file pinned in the NV cache. The minimum IO time observed during an assessment is used for this requirement.
This test is done as part of the WDK tests for validating a hybrid disk but it should also be noted that the test is done by Windows Vista when it boots up and detects a hybrid disk or, in this case, a separated NV cache solution. Devices that do not pass the test will not be used by Windows Vista as NV cache enabled.
In its current form the runtime assessment only requires that disk IOs be aligned to 512 bytes within the file. Given the offset of the volume on the disk and the random nature of the offsets used in the assessment there may be inconsistencies between the alignment of the IOs issued as part of the assessment and the IOs that are issued as part of a realistic user workload. In future versions of the assessment we will look at the algorithm in detail and determine if changes need to be made.
3.5.2 Testing Methodology
Performance testing will primarily use Event Tracing for Windows (ETW) to collect data. ETW leverages existing instrumentation in Windows Vista to gather detailed information about disk IO, process / thread activity, etc. The equivalence testing tools will provide the functionality to collect and analyze the ETW traces such that an automated determination can be made regarding pass or fail of a test. ETW also provides events around power transitions such as standby, hibernate, and shutdown. These events will be used to determine the timing of the operation. Note, that the timing instrumentation is attuned to what would be the user-perceived experience.
Detailed descriptions of each test are given later in the document. The remainder of this section provides a high level overview of each type of test as well as details about the timing goals for each type of test. We first outline the methodology for running these tests. Note that while a percentage based performance goal is ideal as it is oblivious of the absolute performance of the system being tested, a raw time based performance metric is used in conjunction where necessary as Windows imposes timeouts around certain power-transitions.
Finally, note that these tests will be run for both the case write-buffering is enabled on the system and when it is disabled.
3.5.2.1 Methodology
For these tests the goal is to ensure that the user sees acceptable behavior for the operations listed. For each test we execute the operation 5-10 times and compute the average of the times of interest over these runs. The operation itself is run for two cases--when the non volatile cache feature is enabled and when it is disabled. The non-volatile cache feature should be toggled using the Group Policy Object Editor or gpedit.msc.
The following steps disable the non-volatile cache feature:
Launch an elevated command prompt
In the command prompt: type: gpedit.msc
In the popup dialog box with the title Group Policy Object Editor: Click: Local Computer Policy -> Computer Configuration -> Administrative Templates -> System -> Disk NV Cache
Double-click "Turn Off Non Volatile Cache Feature"
Click: enabled
Click: Apply
Click: OK
Issue an unpin all command to the cache
Restart the NV Cache service
The default, or "not configured" setting corresponds to non-volatile cache feature enabled. Note, that another option would be to define the default to be the same system with a typical hard-disk and no separated-cache solution. This is not desirable for two reasons: i) this makes the process of running these tests extremely cumbersome, and ii) we aim to test the performance of the same system when the non-volatile cache feature is disabled to when the feature is enabled, thus comparing the end-user experience in the two cases.
The following steps enable the non-volatile cache feature:
Launch an elevated command prompt
In the command prompt: type: gpedit.msc
In the popup dialog box with the title Group Policy Object Editor: Click: Local Computer Policy -> Computer Configuration -> Administrative Templates -> System -> Disk NV Cache
Double-click "Turn Off Non Volatile Cache Feature"
Click: unconfigured
Click: Apply
Click: OK
Restart the NV Cache service
3.5.2.2 Standby and Hibernate
The goals of the standby and hibernate tests are to ensure not only data consistency during the transitions but also good performance and reliability. The transition into and out of standby and hibernate should be a fast transition not only to give the user a good experience but also to ensure the user's data is protected and that the disk has spun down and the head has been parked shortly after the transition has been initiated. In the event the system is a laptop this allows the user to carry the machine around without the risk of crashing the disk head into the spindle.
From a user's perspective both the timing of going into standby (hibernate) and resuming from standby (hibernate) are of interest. For standby we require that the time to go into standby as well as resume from standby for the non-volatile cache enabled case is no worse than 20% as compared to the non-volatile cache disabled case.
In general, one would expect that the time to resume from hibernate is faster in the non-volatile cache enabled as compared to the non-volatile cache disabled. For this test since our goal is to ensure that the performance is acceptable, we require that the time to resume from hibernate for the former be no worse than 5% as compared to the latter.
3.5.2.3 Shutdown
The goals for the shutdown test are similar to that for the standby/hibernate tests and both the time it takes to shutdown the system as well as the time it takes to boot the system are of interest.
We require that the time it takes to shutdown the system for the non-volatile cache enabled case be no worse than 15s worse as compared to the non-volatile cache disabled case. For boot, in general we would expect the non-volatile cache enabled case to yield smaller boot times. For the purposes of this test since our only goal is to ensure that the performance is acceptable, we require that the boot times for the non-volatile cache enabled case be no worse than 5% as compared to the non-volatile cache disabled case.
4 Tools
4.1 Disk IO Playback Test Tool
Almost all test related activity on the test system will be controlled via this tool. Below is an enumeration and detailed description of all major functionality, command line options, etc. Note the tool and the document will evolve over time so ensure that update to data versions of both are always used.
4.1.1 Overview
In order to provide repeatable and configurable workloads a tool will be provided that allows for the playback of a specific set of IOs as part of a scenario. The input to the tool will be a CSV file that describes the IOs to issue; this is referred to as an "IO Pattern" in this document. The format of the IO pattern file is detailed later in this document.
An IO pattern can be written by hand using a text editor or it can be generated from an ETW trace. ETW traces collect information about disk IO from the partition manager and as such a conversion must be applied to convert the ETW trace to an IO pattern consumable by the tool. The tool will provide options for not only collecting the ETW traces but also for converting them to IO patterns.
Once an IO pattern has been obtained it can be played back with configurable parameters as described below. IO patterns also provide a key extensibility point by which the pattern can be tweaked by hand (to insert sleeps for example) and operations can be inserted manually (such as to initiate a hibernate).
4.1.2 Disk Usage
Since issuing IO patterns that contain writes could be destructive to existing data on the disk the tool will create a large 4 GB file on the system volume and use it for doing the IO in the test.
As part of the test the tool will ensure that the file is contiguous, as such it's recommended for the system volume to have a large amount of free space. This ensures that writes can be issued without corrupting the disk or requiring the disk to be a secondary disk. As such the offsets contained in the IO patterns will be byte offsets relative to the start of the file.
In order to ensure no buffering or caching is done by the OS all IO will be issued as un-buffered.
4.1.3 ETW Tracing Options
The tool will support the collection of ETW traces such that the IO being played back can easily be traced and analyzed. The tracing will be controlled via a command line option and will be off by default.
The tool will also be able to convert an existing ETW trace into an IO pattern. Due to the restrictions on an IO pattern (must use byte offsets and work within a well defined area on the disk) some filtering and munging of the raw ETW data must be performed.
4.1.3.1 Performance Analysis
In addition to collecting traces the tool will provide support for analyzing the ETW traces collected as part of the performance tests.
4.1.4 Command Line Options
This section will evolve as the tool evolves, the most up to date help will always be the command line help provided with the tool.
4.1.4.1 Command Line Help
The tool will support command line help using the "-?" / "/?" option.
4.1.5 IO Pattern Dumps & Options
The tool will support the dumping of ETW traces to IO pattern files as well as the read back of those files into the tool for subsequent testing. The I/O Pattern file should have a csv or comma-separated format. The first line in the file is the command FileSize; there should be no subsequent FileSize commands in the I/O Pattern file. It is assumed that any read/write commands respect the file size specified in the first line else the run is aborted. Also, the tool which issues the command is expected to verify that the file size specified in not greater than the file size on disk. Additionally tags such as "sleep", "hibernate", etc will be supported such that state transitions can be initiated by the tool.
4.1.5.1 IO Pattern File Format
The first string in each line in the I/O pattern file identifies the command and so also the number of subsequent entries in that line. The possible commands are as follows; the strings in italics below correspond to a numerical or ascii value:
FileSize, FileSizeInBytes
FUAFileSize, FileSizeInBytes
WriteCache, BoolEnable
PerformanceTest, NumberOfParallelWriters, NumberOfRuns
Write/FUAWrite, BoolSync, ByteOffset, ByteCount, AsciiChar
Read/FUARead, BoolSync, ByteOffset, ByteCount
Flush
Hibernate, TimeToResumeInSeconds, CrashValue
Standby, TimeToResumeInSeconds, CrashValue
Shutdown, CrashValue
Crash
Except the FileSize/FUAFileSize/WriteCache commands, any number of the above commands could be interspersed arbitrarily. A FUAFileSize command should be present only if it is desired that an additional file be created which is accessed in the write-through mode. FUARead/FUAWrite commands then correspond to IOs to the file opened in the write-through mode.
The WriteCache command specifies a toggling of the disk's write caching in the track buffer. BoolEnable is 1 if it should be enabled or 0 if it should be disabled. This tag in the pattern file translates into a call to the disk with IOCTL_DISK_SET_CACHE_INFORMATION and the WriteCacheEnabled member of the input structure set appropriately.
The PerformanceTest command specifies that the test be a performance test for a power transition. NumberOfParallelWriters specifies the size of the workload on the system before the power transition. Each parallel writer issues write commands to the disk. Each parallel writer writes to its own file so that writes do not go to the same LBA on the disk. A test may have up to a maximum of 15 parallel writers. NumberOfRuns specifies the number of times a system undergoes the power transition for the baseline performance measurements and the actual scenario measurement with the NV Cache feature, respectively. For example, if NumberOfRuns is set to 5, the test would perform a total of 10 power transitions: 5 with the non volatile cache turned off and 5 with the feature turned on.
For the Write command ByteCount number of ASCII character AsciiChar are written to location ByteOffset in the file. The file is assumed to be filled with ASCII '0' to begin with and so also AsciiChar is chosen to be something different from '0'. BoolSync is 1 to indicate synchronous I/O, and 0 for asynchronous I/O. AsciiChar is restricted to the set of values, upper case A-Z.
Sleep corresponds to a call to the "Sleep" API and not to a power transition.
The Hibernate, Standby, Shutdown commands indicate that the system should programmatically be put into a power transition. If the CrashValue tag is 1 then the system will configure the crashing driver to crash when the power transition IRP is received by the crashing driver on the way down into the power transition. If the CrashValue tag is 2 then the system will configure the crashing driver to crash when the power transition IRP is received by the crashing driver on the way up from the power transition. If CrashValue tag is 0 then the system will be configured to automatically resume once the timeout specified by TimeToResumeInMilliseconds has elapsed and the crashing driver will not be used.
The Crash command corresponds to invoking an IOCTL to the crashing driver that crashes the system.
4.1.5.2 IO Pattern Execution
The I/O pattern "executor" creates the I/O Test File filled with character '0' and for the specified FileSizeInBytes for the FileSize command. The executor then simply executes the commands in the I/O Pattern File sequentially. The executor also retrieves the mapping of the file sectors to disk and saves this information in a separate file so that one can identify accesses to the file in the bus-trace.
4.2 Bus Analyzer
In order to accurately test separated NVRAM solutions a bus analyzer is required. This requirement stems from the fact that there is no reliable way to ensure the integrity of the user's data without directly monitoring the contents moving across the bus. Separated NVRAM solutions are designed to be resilient to crashing and when such a situation is detected they attempt to recover and ensure the consistency of the data on the disk. As such in order to properly test these kinds of solutions the consistency of the data on the disk cannot be checked by removing the disk and putting it in another system.
A bus analyzer will be used to ensure the order of IOs that go across the bus is consistent with what has been issued and completed by the test tool.
4.2.1 Bus Analyzer Output / Validation
The output of the bus analyzer will be used to verify that the contents of the disk are consistent and correct for a given test. The Disk IO Playback Tool outlined earlier will have an option to take a parsed and filtered bus trace as input and validate it against the specified test.
The Finisar Bus Doctor RX (RX-108P-BUFF) will be used for the creation and validation of the evaluation tests.
Since all bus analyzers will produce different output the format of the bus trace input to the tool will have to be defined. This will ensure that external parties can produce a tool to convert the output of their bus analyzer to a format that can be consumed by the tool for comparison.
The testing tools will use two methods will be employed to ensure the IOs are being serviced in the correct order that are detailed below.
4.2.1.1 Correlating Bus Trace IOs to issued IOs Using the Offsets
During the execution phase of a given test the test tool gathers information about the location of the file used for the test on the disk. This information combined with knowledge of the IOs issued by the test allows the bus trace IOs to match to their corresponding IOs issued by the test using the offset.
While any bus-analyzer could be used to trace the IO bus activity, the bus-trace needs to be processed and should have a certain format in order for it to be consumed by the tool used for verifying the IOs. For verification purposes only the Read, Write and Flush commands seen in the trace are of interest. The desired format for these three commands of interest is as follows; the strings in italics below correspond to a numerical value:
READ, LbaOffset, LbaCount
WRITE, LbaOffset, LbaCount
FLUSH
In the above, LbaOffset corresponds to the LBA offset to which the IO (read or write) was issued and LbaCount indicates the numbers if LBAs requested. READ and WRITE indicate whether the IO was a read or a write. The FLUSH command corresponds to a flush command observed in the bus-trace.
4.3 Faulty Driver
The faulty driver is a kernel mode driver that is able to bug check the system at a specified time. This driver will read a time value from the registry that corresponds to the time delay in milliseconds from driver load until the driver should bug check. The driver will maintain a count via the registry of the number of times it has been loaded and ensure that it does not bug check more than the number of times specified (also a registry key). This allows a system with the driver present to be (somewhat) easily recovered.
Additional modes will be required to signal that the driver should crash at well defined transitions, such as during the resume from standby. These will all be able to be toggled using the command line test tool.
Additionally an IOCTL will be exposed from the driver that will allow the system to be bug checked on demand.
Options will be provided with the test tool to install and uninstall the crashing driver.
4.4 Power Loss
Testing of scenarios that involve power loss will be a manual process in the first iteration of these tests. Power loss is defined as removing power from a system that contains no other power source that can be used for powering the system for general use, such as a laptop battery. Once power has been removed from the system it is imperative that the disk is not removed from the system and in general that the hardware is not tampered with in any way.
Power loss scenarios can be automated in the future using programmable power strings to ensure repeatability as well as reduce the amount of manual activity required to run the tests.
5 Test Details
5.1 Overview
5.1.1 System Preparation
To ensure that background system activity does not influence the test results we suggest disabling some components - the details on how this can be achieved are included in the README which is distributed with the test suite.
5.1.2 Test Initialization
In order to ensure the start of a test can be easily recognized in the output of the bus analyzer the test will issue a well defined pattern of IOs. This pattern will consist of 100 512 byte IOs issued to the 512 byte aligned middle-offset of the file. Note, that this is step is necessary only for the tests which require correlating the issued IOs to that seen in a bus trace.
5.1.3 IO Patterns
The IO patterns of interest for these test cases revolve around write activity. As such the initial set of test cases will contain a straightforward mixture of writes and flushes designed to provide an easily reproducible and verifiable IO pattern. The writes and flushes will be intermixed with power transitions and power loss events.
Each test will outline the pattern of IO issued by the test application as well as the required results as observed from the bus analyzer.
5.1.4 Power Transitions and Crashes
The power transitions that will be used in this version of the tests include the following:
Suspend to Standby (S3)
Suspend to Hibernate (S4)
Shutdown (S5)
Kernel Mode Crash / Bugcheck
Unexpected power loss
Most of the above will be issued programmatically and will not require any manual actions. As stated earlier the power loss transitions will require manual intervention both to remove and restore power.
5.2 Synthetic Data Consistency / Reliability Test Cases
5.2.1 Overview
In general the tests consist of a well defined set of IO followed by some action. The IO issued by each test and the expected result is given below for each test in detail. There is often significant overlap between tests of a similar category but rather than generalize and potentially introduce inaccuracies or assumptions each test is always described in detail.
The goal of these tests is to focus narrowly on a specific set of difficult problems and ensure they are handled correctly. This automatic NVCache flush command, however, is not necessary and so will not be issued when running the performance tests (described later).
Finally, note that the sequence of commands listed for each test case below is merely representative, The actual number of writes issued between two flushes, the IO sizes, the number of non-crashing power transitions etc. can vary a little
In the following, for saving space we have shown a sample workload with 4 writes and two flushes. The IOPattern file used in the test suite has 100 writes, 10 flushes.
In every IOPattern file, there will be no more than one type of power transition, e.g. an IOPattern file would not contain both standby and shutdown. For a test case containing shutdown, there is one shutdown command in the entire IOPattern file, and the shutdown command is always the last command in the file. For hibernate and standby tests, there are two power transition commands in each test case. An exception for the standby and hibernate tests is the power loss tests; in this case the IOPattern file contains only one standby or hibernate command. The last power transition in the IOPattern file may contain crash (with the crash flag set), depending on the particular test case.
5.2.2 Basic Write / Flush Test
The goal of the Basic Write / Flush test is to ensure that in the absence of a proper power transition that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
A call to FlushFileBuffers
5.2.2.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 completes that the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section is 5.2.2.1.io.txt.
5.2.2.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 completes that the data for Steps 4 and 5 has been observed in the trace in any order.
The test case files for this section are 5.2.2.2.io.txt and 5.2.2.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.2.3 Basic Write / Flush Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
A call to the crashing driver to crash the system.
The system will be rebooted and allowed to boot completely into the OS which could trigger a NVRAM cache recovery operation if needed.
5.2.2.3.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 7 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.2.3.1.io.txt and 5.2.2.3.1.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.2.3.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 7 the data for Step 4 and Step 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.2.3.2.io.txt and 5.2.2.3.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.2.4 Basic Write / Flush Test with Power Loss
This permutation of the test inserts a sudden power loss into the test. The goal of this test is to ensure that the proper ordering is maintained in lieu of a sudden power loss and a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
Power will be removed from the system.
The system will be rebooted and allowed to boot completely into the OS which could trigger a NVRAM cache recovery operation if needed.
5.2.2.4.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 7 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.2.4.1.io.txt and 5.2.2.4.1.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.2.4.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 7 the data for Step 4 and Step 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.2.4.2.io.txt and 5.2.2.4.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.2.5 Basic Write / Flush Test with Power Loss 2
This permutation of the test inserts a sudden power loss into the test as well as a power loss during the potential recovery step. The goal of this test is to ensure that the proper ordering is maintained in lieu of a sudden power loss and a power loss during a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
Power will be removed from the system.
The system will be booted.
Prior the system booting into the OS power will be removed from the system.
- Note this step is a bit undefined right now as there is no known reliable way to detect if this recovery is going on. There may be activity on the bus trace that can signify it is happening; it is an area that will need to be investigated.
The system will be booted and allowed to boot completely into the OS.
5.2.2.5.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 9 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.2.5.1.io.txt and 5.2.2.5.1.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.2.5.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 9 the data for Step 4 and Step 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.2.5.2.io.txt and 5.2.2.5.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.3 Standby / Resume Test
The goal of the Standby / Resume test is to ensure that in the presence of a proper power transition that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby.
The system will be resumed.
A call to FlushFileBuffers
5.2.3.1.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.3.1.1.io.txt.
5.2.3.1.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7.
After Step 8 completes that the data for Steps 4 and 5 has been observed in the trace in any order.
The test case file for this section is 5.2.3.1.2.io.txt.
5.2.3.2 Standby / Resume Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation. The crashing driver will be set up to crash one time immediately after being notified of the resume operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset.
4 KB write of all "D"s to offset Z.
The system will suspend into standby
The system will be resumed and the driver will crash the system during the resume.
The system will be booted completely into the OS and the driver will not crash it this time.
5.2.3.2.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.3.2.1.io.txt and 5.2.3.2.1.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.3.2.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7.
After Step 8 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.3.2.2.io.txt and 5.2.3.2.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.3.3 Standby / Resume Test with Crash 2
This permutation of the test inserts a call to the crashing driver to crash the system during the standby operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby and the crashing driver will crash the system before the transition is complete.
. The system will be booted completely into the OS and the driver will not crash it this time.
5.2.3.3.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order.
After Step 7 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.3.3.1.io.txt.
5.2.3.3.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order.
After Step 7 completes the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.3.3.2.io.txt and 5.2.3.3.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.3.4 Standby / Resume Test with Power Loss
This permutation of the test inserts an unexpected power loss during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a recovery operation.
The test will issue the following set of IO
. 4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby
The system will be resumed and the power will be removed during the resume.
Power will be restored and the system will be booted completely into the OS.
5.2.3.4.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.3.4.1.io.txt and 5.2.3.4.1.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.3.4.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7.
After Step 8 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.3.4.2.io.txt and 5.2.3.4.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.3.5 Standby / Resume Test with Power Loss 2
This permutation of the test inserts an unexpected power loss during the resume operation as well as a power loss during the recovery operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a power loss during a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby
The system will be resumed and the power will be removed during the resume.
Power will be restored and the system will be booted. Prior the system booting into the OS power will be removed from the system.
Power will be restored and the system will be booted completely into the OS.
5.2.3.5.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7 and Step 8.
After Step 9 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.3.5.1.io.txt.
5.2.3.5.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7 and Step 8.
After Step 9 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.3.5.2.io.txt and 5.2.3.5.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.4 Hibernate / Resume Test
The goal of the Hibernate / Resume test is to ensure that in the presence of a proper power transition that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate.
The system will be resumed.
A call to FlushFileBuffers
5.2.4.1.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.4.1.1.io.txt.
5.2.4.1.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7.
After Step 8 completes that the data for Steps 4 and 5 has been observed in the trace in any order.
The test case file for this section is 5.2.4.1.2.io.txt.
5.2.4.2 Hibernate / Resume Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation. The crashing driver will be set up to crash one time immediately after being notified of the resume operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate.
The system will be resumed and the driver will crash the system during the resume.
The system will be booted completely into the OS and the driver will not crash it this time.
5.2.4.2.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.4.2.1.io.txt.
5.2.4.2.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7.
After Step 8 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.4.2.2.io.txt and 5.2.4.2.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.4.3 Hibernate / Resume Test with Crash 2
This permutation of the test inserts a call to the crashing driver to crash the system during the hibernate operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate and the crashing driver will crash the system before the hibernate operation completes.
The system will be booted completely into the OS and the driver will not crash it this time.
5.2.4.3.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order.
After Step 7 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.4.3.1.io.txt.
5.2.4.3.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order.
After Step 7 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.4.3.2.io.txt and 5.2.4.3.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.4.4 Hibernate / Resume Test with Power Loss
This permutation of the test inserts an unexpected power loss during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate
The system will be resumed and the power will be removed during the resume.
Power will be restored and the system will be booted completely into the OS.
5.2.4.4.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.4.4.1.io.txt.
5.2.4.4.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7.
After Step 8 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.4.4.2.io.txt and 5.2.4.4.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.4.5 Hibernate / Resume Test with Power Loss 2
This permutation of the test inserts an unexpected power loss during the resume operation as well as a power loss during the recovery operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a power loss during a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate
The system will be resumed and the power will be removed during the resume.
Power will be restored and the system will be booted. Prior the system booting into the OS power will be removed from the system.
Power will be restored and the system will be booted completely into the OS.
5.2.4.5.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This same requirement holds true for Step 7 and Step 8.
After Step 9 the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.4.5.1.io.txt.
5.2.4.5.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7 and Step 8.
After Step 9 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.4.5.2.io.txt and 5.2.4.5.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.5 Shutdown / Restart Test
The goal of the Shutdown / Restart test is to ensure that in the presence of a proper shutdown and reboot that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown cleanly.
5.2.5.1.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 6 completes that the data for Steps 4 and 5 has been observed in the trace in order.
The test case file for this section is 5.2.5.1.1.io.txt.
5.2.5.1.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 completes that the data for Steps 4 and 5 has been observed in the trace in any order.
The test case file for this section is 5.2.5.1.2.io.txt.
5.2.5.2 Shutdown / Restart Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system during the shutdown operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation. The crashing driver will be set up to crash one time during the shutdown process.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown.
During the shutdown the crashing driver will crash the system.
The system will be booted completely into the OS.
5.2.5.2.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 7 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.5.2.1.io.txt
5.2.5.2.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 6 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This same requirement holds true for Step 7.
After Step 8 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.5.2.2.io.txt and 5.2.5.2.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.5.3 Shutdown / Restart Test with Power Loss
This permutation of the test inserts an unexpected power loss during the shutdown operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown
Power will be removed from the system before the shutdown is complete.
Power will be restored and the system will be booted completely into the OS.
5.2.5.3.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 7 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order.
After Step 8 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.5.3.1.io.txt.
5.2.5.3.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 7 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order.
After Step 8 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.5.3.2.io.txt and 5.2.5.3.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.5.4 Shutdown / Restart Test with Power Loss 2
This permutation of the test inserts an unexpected power loss during the shutdown operation as well as a power loss during the recovery operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a power loss during a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown
Power will be removed from the system before the shutdown is complete.
Power will be restored and the system will be booted. Prior the system booting into the OS power will be removed from the system.
Power will be restored and the system will be booted completely into the OS.
5.2.5.4.1 Valid Results with Write Buffering Disabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in order.
After Step 7 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in order. This requirement also holds true for Step 8.
After Step 9 the data for Steps 4 and 5 has been observed in the trace in order.
The test case files for this section are 5.2.5.4.1.io.txt
5.2.5.4.2 Valid Results with Write Buffering Enabled
After Step 3 completes that the data for Steps 1 and 2 has been observed in the trace in any order.
After Step 7 the data for Step 4 or Step 4 and Step 5 may have been observed in the trace in any order. This requirement also holds true for Step 8.
After Step 9 completes that the data for Steps 4 and 5 may have been observed in the trace in any order. It is valid for the data in Steps 4 and 5 to never be observed in the trace for this test.
The test case files for this section are 5.2.5.4.2.io.txt and 5.2.5.4.2.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.6 FUA Test Cases
In this section, we outline tests for the case the application issues interspersed IOs to two files one of which is accessed in the write-through mode and the other in the write-back mode[2]. These tests need to be run only for the case the write buffering is enabled on the system. For the case write buffering is disabled the test cases in the previous sections suffice.
There is a FUA test case corresponding to each of tests listed in Sections 5.2.2 through 5.2.4 above. The ordering of IOs for these test cases essentially follows from the discussion in Section 3.4.4. Rather then repeat each of the test scenarios listed above we elaborate only a sample test case. The ordering requirements for the remaining test cases can be inferred from this example.
In the following, write and FUAwrite respectively, correspond to writes being issued o two different files opened in the write-through and write-back mode, respectively,
NOTE: FUA Test Cases are NOT part of the LOGO requirements. The Separated Cache Solution is not required to run or pass any FUA Tests.
5.2.6.1 Basic Write / Flush Test
The goal of the Basic Write / Flush test is to ensure that in the absence of a proper power transition that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.1.fua.io.txt and 5.2.6.1.fua.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.6.2 Basic Write / Flush Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
A call to the crashing driver to crash the system.
The system will be rebooted and allowed to boot completely into the OS which could trigger a NVRAM cache recovery operation if needed.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.2.fua.io.txt and 5.2.6.2.fua.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.6.3 Basic Write / Flush Test with Power Loss
This permutation of the test inserts a sudden power loss into the test. The goal of this test is to ensure that the proper ordering is maintained in lieu of a sudden power loss and a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB FUAwrite of all "S"s to offset L.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
Power will be removed from the system.
The system will be rebooted and allowed to boot completely into the OS which could trigger a NVRAM cache recovery operation if needed.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.3.fua.io.txt and 5.2.6.3.fua.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.6.4 Standby / Resume Test
The goal of the Standby / Resume test is to ensure that in the presence of a proper power transition that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby.
The system will be resumed.
A call to FlushFileBuffers
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case file for this section is 5.2.6.4.fua.io.txt.
5.2.6.5 Standby / Resume Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation. The crashing driver will be set up to crash one time immediately after being notified of the resume operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby
The system will be resumed and the driver will crash the system during the resume.
The system will be booted completely into the OS and the driver will not crash it this time.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case file for this section is 5.2.6.5.fua.io.txt.
5.2.6.6 Standby / Resume Test with Crash 2
This permutation of the test inserts a call to the crashing driver to crash the system during the standby operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby and the crashing driver will crash the system before the transition is complete.
The system will be booted completely into the OS and the driver will not crash it this time.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.6.fua.io.txt and 5.2.6.6.fua.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.6.7 Standby / Resume Test with Power Loss
This permutation of the test inserts an unexpected power loss during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby
The system will be resumed and the power will be removed during the resume.
Power will be restored and the system will be booted completely into the OS.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case file for this section is 5.2.6.7.fua.io.txt.
5.2.6.8 Standby / Resume Test with Power Loss 2
This permutation of the test inserts an unexpected power loss during the resume operation as well as a power loss during the recovery operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a power loss during a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will suspend into standby
The system will be resumed and the power will be removed during the resume.
Power will be restored and the system will be booted. Prior the system booting into the OS power will be removed from the system.
Power will be restored and the system will be booted completely into the OS.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.8.fua.io.txt and 5.2.6.8.fua.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.6.9 Hibernate / Resume Test
The goal of the Hibernate / Resume test is to ensure that in the presence of a proper power transition that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate.
The system will be resumed.
A call to FlushFileBuffers
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.9.fua.io.txt.
5.2.6.10 Hibernate / Resume Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation. The crashing driver will be set up to crash one time immediately after being notified of the resume operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate.
The system will be resumed and the driver will crash the system during the resume.
The system will be booted completely into the OS and the driver will not crash it this time.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.10.fua.io.txt and 5.2.6.10.fua.ff.io.txt. The former file has a flush frequency of 10 writes and the latter file has a flush frequency of 2 writes.
5.2.6.11 Hibernate / Resume Test with Crash 2
This permutation of the test inserts a call to the crashing driver to crash the system during the hibernate operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation.
The test will issue the following set of IO
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate and the crashing driver will crash the system before the hibernate operation completes.
The system will be booted completely into the OS and the driver will not crash it this time.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case file for this section is 5.2.6.11.fua.io.txt.
5.2.6.12 Hibernate / Resume Test with Power Loss
This permutation of the test inserts an unexpected power loss during the resume operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate.
The system will be resumed and the driver will crash the system during the resume.
Power will be restored and the system will be booted completely into the OS.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.12.fua.io.txt.
5.2.6.13 Hibernate / Resume Test with Power Loss 2
This permutation of the test inserts an unexpected power loss during the resume operation as well as a power loss during the recovery operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a power loss during a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be put into hibernate.
The system will be resumed and the driver will crash the system during the resume.
Power will be restored and the system will be booted. Prior the system booting into the OS power will be removed from the system.
Power will be restored and the system will be booted completely into the OS.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case file for this section is 5.2.6.13.fua.io.txt.
5.2.6.14 Shutdown / Restart Test
The goal of the Shutdown / Restart test is to ensure that in the presence of a proper shutdown and reboot that the correct ordering of writes is maintained and that flushes are obeyed correctly.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown cleanly.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case file for this section is 5.2.6.14.fua.io.txt.
5.2.6.15 Shutdown / Restart Test with Crash
This permutation of the test inserts a call to the crashing driver to crash the system during the shutdown operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a crash and a recovery operation. The crashing driver will be set up to crash one time during the shutdown process.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown.
During the shutdown the crashing driver will crash the system.
The system will be booted completely into the OS.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case file for this section is 5.2.6.15.fua.io.txt. The frequent flush test case is 5.2.6.15.fua.ff.io.txt.
5.2.6.16 Shutdown / Restart Test with Power Loss
This permutation of the test inserts an unexpected power loss during the shutdown operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown.
Power will be removed from the system before the shutdown is complete.
Power will be restored and the system will be booted completely into the OS.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.16.fua.io.txt.
5.2.6.17 Shutdown / Restart Test with Power Loss 2
This permutation of the test inserts an unexpected power loss during the shutdown operation as well as a power loss during the recovery operation. The goal of this test is to ensure that the proper ordering is maintained in lieu of a power loss and a power loss during a recovery operation.
The test will issue the following set of IO
4 KB FUAwrite of all "P"s to offset I.
4 KB write of all "A"s to offset X.
4 KB write of all "B"s to offset Y.
4 KB FUAwrite of all "Q"s to offset J.
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
4 KB FUAwrite of all "R"s to offset K.
A call to FlushFileBuffers
4 KB write of all "C"s to offset W.
4 KB write of all "D"s to offset Z.
The system will be shutdown.
Power will be removed from the system before the shutdown is complete.
Power will be restored and the system will be booted. Prior the system booting into the OS power will be removed from the system.
Power will be restored and the system will be booted completely into the OS.
Valid results:
None of the data for steps 2 through 7 are observed before the data for step 1 has been observed in the trace.
When the IO corresponding to step 4 is observed in the trace none of data for step 5 through 7 has been observed in the trace. The data for steps 2 and 3 may or may not have been observed in the trace.
When the data for step 7 is observed in the trace the data for step 1 and step 4 has been observed in the trace, the data for steps 2, 3, 5 and 6 may or may not have been observed in the trace.
When step 8 completes all the data for steps 1 through 7 has been observed in the trace.
The test case files for this section are 5.2.6.17.fua.io.txt.
5.3 Synthetic Performance Test Cases
As with the previous tests each test case below is given in detail so as to ensure there is no confusion or ambiguity about the structure of the test or the expected results.
All performance testing is done with write caching in the disk's track buffer enabled. The methodology for these tests is similar to that outlined in Section 3.5.2.1. Also, similarly we compare the average timing for the operations listed below for the hybrid case to the default case over 5-10 runs for each case.
While it would be desirable to provide some representative performance numbers for these tests the unavailability of prototypes in the market prevents us from doing so. These goals have been designed to ensure an acceptable user experience and moreover, the fact that the goals are relative in nature makes providing representative performance numbers unnecessary. Here, by relative we refer to the fact that the performance goals are not absolute and compare the performance of the same system when the non-volatile cache feature is enabled to when it is disabled.
Finally, note that these tests will be run for both the case with write-buffering enabled on the system and the case with write-buffering disabled.
5.3.1 Standby
The concerns around standby revolve around there being a sufficient amount of dirty data in NV cache such that when the power transition is initiated a significant amount of time is required to flush out the dirty data. This would not only greatly reduce the quality of that experience for the user but also puts their data at risk if the disk is required to continue spinning much longer after the transition is initiated (i.e. the lid closed).
For standby we require that the time to go into standby as well as resume from standby for the non-volatile cache enabled case is no worse than 20% as compared to the non-volatile cache disabled case.
5.3.1.1 Suspend with Heavy Write Activity 1
The goal of this test is to accumulate a significant amount of unique dirty data shortly before transitioning to standby. A flush happens for every ten writes in this test. The test script file for this test case with write caching disabled is 5.3.1.1.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.1.1.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same. Passing the tests requires that the system completely enter standby within a specified period of time.
The test is comprised of the following steps:
Steps 1 to 6 measure standby and resume time with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
Initiate a standby power transition and start a timer which wakes up the system after 1 minute (i.e. the system would stay in the standby state for 1 minute).
Delay for 2 minutes after the systems resumes from standby
Extract standby and resume time from the system's Performance Diagnostics event log
Repeat steps 2. to 5. five times
Steps 7 to 12 measure standby and resume time with non-volatile cache enabled
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
NOTE: These will be the same IO which were issued in Step 1.
Initiate a standby power transition and start a timer which wakes up the system after 1 minute. (i.e. the system would stay in the standby state for 1 minute).
Delay for 2 minutes after the systems resumes from standby.
Extract standby and resume time from the system's Performance Diagnostics event log.
Repeat steps 8-11 five times.
Calculate the average for standby and resume duration measurements with the non-volatile cache turned on and turned off respectively.
5.3.1.1.1 Passing this test requires that going into standby and resuming (i.e. displaying of the desktop) completes within a set amount of time. The test fails if any of the following conditions is true:
The average time for the system to go into standby for the non-volatile cache enabled case is 20% above the non-volatile cache disabled case
The average time for the system to resume from standby for the non-volatile cache enabled case is 20% above the non-volatile cache disabled case
5.3.1.2 Suspend with Heavy Write Activity 2
The goal of this test is to accumulate a significant amount of unique dirty data shortly before issuing an explicit flush prior to transitioning to standby. A flush happens for every two writes in this test--the difference between this test case and 5.3.1.1. The test script file for this test case with write caching disabled is 5.3.1.2.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.1.2.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same. Passing the test requires that the system completely enter standby within a specified period of time.
The test is comprised of the following steps:
Steps 1 to 7 measure standby and resume time with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a FlushFileBuffers call.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
Initiate a standby power transition and start a timer which wakes up the system after 1 minute (i.e. the system would stay in the standby state for 1 minute).
Delay for 2 minutes after the systems resumes from standby.
Extract standby and resume time from the system's Performance Diagnostics event log.
Repeat steps 2-6 five times.
Steps 8 to 14 measure standby and resume time with non-volatile cache enabled.
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a FlushFileBuffers call.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
NOTE: These will be the same IO which were issued in Step 1.
Initiate a standby power transition and start a timer which wakes up the system after 1 minute. (For example, the system would stay in the standby state for 1 minute.)
Delay for 2 minutes after the systems resumes from standby.
Extract standby and resume time from the system's Performance Diagnostics event log.
Repeat steps 9-13 five times
Calculate the average for standby and resume duration measurements with the non-volatile cache turned on and turned off respectively.
5.3.1.2.1 Passing this test requires that the going into standby and resuming (i.e. displaying of the desktop) completes within a set amount of time. The test fails if any of the following conditions is true:
The average time for the system to go into standby for the non-volatile cache enabled case is 20% above the non-volatile cache disabled case
The average time for the system to resume from standby for the non-volatile cache enabled case is 20% above the non-volatile cache disabled case
5.3.2 Hibernate
The concerns around hibernate revolve around there being a sufficient amount of dirty data in NV cache such that when the power transition is initiated a significant amount of time is required to flush out the dirty data. This would not only greatly reduce the quality of that experience for the user but also puts their data at risk if the disk is required to continue spinning much longer after the transition is initiated (i.e. the lid closed).
In general, one would expect that the time to resume from hibernate is faster in the non-volatile cache enabled case as compared to the non-volatile cache disabled case. For this test, we require that the time to resume from hibernate for the former be no worse than 5% as compared to the latter.
5.3.2.1 Hibernate with Heavy Write Activity 1
The goal of this test is to accumulate a significant amount of unique dirty data shortly before transitioning to hibernate. A flush happens for every ten writes in this test. The test script file for this test case with write caching disabled is 5.3.2.1.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.2.1.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same. Passing the test requires that the system completely hibernate within a specified period of time.
The test is comprised of the following steps:
Steps 1 to 6 measure hibernate resume time with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
Initiate a hibernate power transition and start a timer which wakes up the system after 1 minute (for example, the system would stay in the hibernate state for 1 minute).
Delay for 2 minutes after the systems resumes from hibernate.
Extract hibernate resume time from the system's Performance Diagnostics event log.
Repeat steps 2-5 five times.
Steps 7 to 12 measure hibernate resume time with non-volatile cache enabled.
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
NOTE: These will be the same IO which were issued in Step 1.
Initiate a hibernate power transition and start a timer which wakes up the system after 1 minute (for example, the system would stay in the hibernate state for 1 minute).
Delay for 2 minutes after the systems resumes from hibernate.
Extract hibernate resume time from the system's Performance Diagnostics event log.
Repeat steps 8-11 five times.
Calculate the average for hibernate resume duration measurements with the non-volatile cache turned on and turned off respectively.
5.3.2.1.1 Passing this test requires that hibernate resume (i.e. displaying of the desktop) completes within a set amount of time. The test fails if both the following conditions are true:
The average time for the system to resume from hibernate for the non-volatile cache enabled case is 5% above the non-volatile cache disabled case.
The average time for the system to resume from hibernate for the non-volatile cache enabled case exceeds the non-volatile cache disabled case by over 2 seconds.
5.3.2.2 Hibernate with Heavy Write Activity 2
The goal of this test is to accumulate a significant amount of unique dirty data shortly before issuing an explicit flush prior to transitioning to hibernate. A flush happens for every two writes in this test--the difference between this test case and 5.3.2.1. The test script file for this test case with write caching disabled is 5.3.2.2.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.2.2.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same. Passing the test requires that the system completely hibernate within a specified period of time.
The test is comprised of the following steps:
Steps 1 to 6 measure hibernate resume time with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a FlushFileBuffers call.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
Initiate a hibernate power transition and start a timer which wakes up the system after 1 minute (for example, the system would stay in the hibernate state for 1 minute).
Delay for 2 minutes after the systems resumes from hibernate.
Extract hibernate resume time from the system's Performance Diagnostics event log.
Repeat steps 2-6 five times.
Steps 8 to 14 measure hibernate resume time with non-volatile cache enabled.
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a FlushFileBuffers call.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
NOTE: These will be the same IO which were issued in Step 1.
Initiate a hibernate power transition and start a timer which wakes up the system after 1 minute (for example the system would stay in the hibernate state for 1 minute).
Delay for 2 minutes after the systems resumes from hibernate.
Extract hibernate resume time from the system's Performance Diagnostics event log.
Repeat steps 9-13. five times.
Calculate the average for hibernate resume duration measurements with the non-volatile cache turned on and turned off respectively.
5.3.2.2.1 Passing this test requires that hibernate resume (i.e. displaying of the desktop) completes within a set amount of time. The test fails if both the following conditions are true:
The average time for the system to resume from hibernate for the non-volatile cache enabled case is 5% above the non-volatile cache disabled case.
The average time for the system to resume from hibernate for the non-volatile cache enabled case exceeds the non-volatile cache disabled case by over 2 seconds.
5.3.3 Shutdown
The concerns around shutdown revolve around there being a sufficient amount of dirty data in NV cache such that when the power transition is initiated a significant amount of time is required to flush out the dirty data. This would not only greatly reduce the quality of that experience for the user but also puts their data at risk if the disk is required to continue spinning much longer after the transition is initiated (i.e. the lid closed).
We require that the time it takes to shutdown the system for the non-volatile cache enabled case be no worse than 15s worse as compared to the non-volatile cache disabled case. For boot, in general we would expect the non-volatile cache enabled case to yield smaller boot times. For the purposes of this test, we require that the boot times for the non-volatile cache enabled case be no worse than 5% as compared to the non-volatile cache disabled case.
5.3.3.1 Shutdown with Heavy Write Activity 1
The goal of this test is to accumulate a significant amount of unique dirty data shortly before shutting down the system. A flush happens for every ten writes in this test. The test script file for this test case with write caching disabled is 5.3.3.1.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.3.1.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same. Passing the test requires that the system shutdown within a specified period of time.
The test is comprised of the following steps:
Steps 1 to 6 measure shutdown and boot time with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
Initiate a shutdown power transition
Delay for 2 minutes after the systems boots
Extract shutdown and boot time from the system's Performance Diagnostics event log
Repeat steps 2. to 5. five times
Steps 7 to measure shutdown and boot time with non-volatile cache enabled
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
NOTE: These will be the same IO which were issued in Step 1
Initiate a shutdown power transition.
Delay for 2 minutes after the systems reboots.
Extract shutdown and boot times from the system's Performance Diagnostics event log.
Repeat steps 8-11 five times.
Calculate the average for shutdown and boot duration measurements with the non-volatile cache turned on and turned off respectively.
5.3.3.1.1 Passing this test requires that shutdown and boot (i.e. displaying of the desktop) completes within a set amount of time. The test fails if any of the following conditions is true:
The average time for the system to shutdown for the non-volatile cache enabled case is 15 seconds longer than the non-volatile cache disabled case
The average time for the system to boot for the non-volatile cache enabled case is 5% above the non-volatile cache disabled case
5.3.3.2 Shutdown with Heavy Write Activity 2
The goal of this test is to accumulate a significant amount of unique dirty data shortly before issuing an explicit call to flush the track buffer followed by a shutdown. A flush happens for every two writes in this test--the difference between this test case and 5.3.3.2. The test script file for this test case with write caching disabled is 5.3.3.2.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.3.2.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same. Passing the test requires that the system shutdown within a specified period of time.
The test is comprised of the following steps:
Steps 1 to 7 measure shutdown and boot time with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a Flush command.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
Initiate a shutdown power transition.
Delay for 2 minutes after the systems boots.
Extract shutdown and boot time from the system's Performance Diagnostics event log.
Repeat steps 2-6 five times.
Steps 8 to 14 measure shutdown and boot time with non-volatile cache enabled
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a Flush command.
Issue 64 MB of random IO using 64 KB IOs to unique offsets. i.e. no offset IO will be issued to the same offset twice in this test.
NOTE: These will be the same IO which were issued in Step 1.
Initiate a shutdown power transition.
Delay for 2 minutes after the systems reboots.
Extract shutdown and boot times from the system's Performance Diagnostics event log.
Repeat steps 9-13 five times.
Calculate the average for shutdown and boot duration measurements with the non-volatile cache turned on and turned off respectively.
5.3.3.2.1 Passing this test requires that shutdown and boot (i.e. displaying of the desktop) completes within a set amount of time. The test fails if any of the following conditions is true:
The average time for the system to shutdown for the non-volatile cache enabled case is 15 seconds longer than the non-volatile cache disabled case
The average time for the system to boot for the non-volatile cache enabled case is 5% above the non-volatile cache disabled case
5.3.4 General Use
The concerns around general use performance are vast and this set of tests is most likely to grow over time as engineering samples for separated NV cache solutions are received and can be evaluated. The goal of these tests is reproduce common problematic behavior and ensure that in the face of said behavior a separated NV cache solution can deliver an high performance experience.
5.3.4.1 Heavy Random Write Activity with Read
The goal of this test is to ensure in the presence of a significant amount of write activity that IOs are not blocked for long periods of time while book keeping and other flush activities associated with the NV cache are performed. The test script file for this test case with write caching disabled is 5.3.4.1.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.4.1.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same.
The test is comprised of the following steps:
Steps 1 to 7 measure read time during flush with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a Flush command.
Issue 64 MB of random IO using sizes randomly chosen from 4 KB to 64 KB. No effort is made to ensure the offsets are unique on thread 1.
Issue a FlushFileBuffers call on thread 1.
Issue a random read on thread 2.
Measure IO time of the random read from step 5.
Repeat steps 2-6 six times.
Steps 8 to 14 measure read time during flush with non-volatile cache enabled
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a Flush command.
Issue 64 MB of random IO using sizes randomly chosen from 4 KB to 64 KB. No effort is made to ensure the offsets are unique on thread 1.
Issue a FlushFileBuffers call on thread 1.
Issue a random read on thread 2.
Measure IO time of the random read from step 12.
Repeat steps 9-13. six times.
Calculate the median IO time for each run in steps with the non-volatile cache turned on and turned off respectively.
5.3.4.1.1 Passing this test requires that the random read on thread 2 issued in Step 3 complete within a set amount of time. The test fails if any of the following conditions is true:
- The median IO time for the non-volatile cache enabled case is 10ms above the non-volatile cache disabled case
5.3.4.2 Heavy Random Write Activity
The goal of this test is to ensure in the presence of a significant amount of write activity that IOs are not blocked for long periods of time while book keeping and other flush activities associated with the NV cache are performed. The test script file for this test case with write caching disabled is 5.3.4.2.WC0.io.txt. The test script file for this test case with write caching enabled is 5.3.4.2.WC1.io.txt. The passing criteria for both the write cache enabled and disabled cases are the same.
The test is comprised of the following steps:
Steps 1 to 5 measure IO times with 15 threads writing the same IO pattern to different files with non-volatile cache disabled
Disable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a Flush command.
Each of the 15 threads issues 64 MB of random IO using 64 KB IOs to unique offsets. Each thread will write to its own file using the same IO pattern. i.e. no offset IO will be issued to the same offset twice in this test.
Measure IO time of each IO.
Repeat steps 2-4 six times.
Steps 6 to 10 measure IO times with 15 threads writing to the same offsets in different files with non-volatile cache enabled.
Enable non-volatile cache as outlined in section 3.5.2.1 earlier in this specification.
Issue a Flush command.
Each of the 15 threads issues 64 MB of random IO using 64 KB IOs to unique offsets. Each thread will write to its own file using the same IO pattern. i.e. no offset IO will be issued to the same offset twice in this test.
Measure IO time of each IO.
Repeat steps 7-9 six times.
Calculate the median IO time for each run with the non-volatile cache turned on and turned off respectively.
5.3.4.2.1 Passing this test requires that the IOs complete within a set amount of time. The test fails if any of the following conditions is true:
- The median IO time for the non-volatile cache enabled case is 10ms above the non-volatile cache disabled case
5.4 Realistic User Performance Test Cases
The tests outlined so far have focused on synthetic workloads and were geared towards testing the behavior for baseline scenarios. In this section, we outline some realistic user performance test cases. These are geared towards ensuring that a separated-cache solution does not result in a performance degradation for real-world workload scenarios. We first outline the real-world scenarios and then outline our methodology for running these tests.
5.4.1 Real-world Scenarios
Each of the scenarios listed below correspond to a real-world scenario which results in some amount of write activity, with or without interactive user inputs, on a machine. For each scenario below we provide some statistics as observed in a trace when the scenario was run. The scenarios range in duration from one to three minutes and exhibit low to moderate CPU activity. The disk utilization over the duration of the trace is observed to be less than 15% in each scenario.
5.4.1.1 Document download
This scenario involves downloading a couple of documents from the internet. The observed statistics are as shown in the following table.
Trace Duration |
80 s |
System Utilization Statistics |
CPU Util. 18% and Disk Util. 9% |
I/O Count Statistics |
2378 I/Os with 27.5 % Reads |
I/O Size Statistics |
70.26 MB with 76.35 % Reads |
Flush Count |
85 |
Test Case File: DocumentDownload.IOPattern.txt
5.4.1.2 Image editing
This scenario involves editing images and saving them as they are viewed in a slide show. The editing was limited to adjusting the color and/or the exposure. The observed statistics are as shown in the following table.
Trace Duration |
138 s |
System Utilization Statistics |
CPU Util. 19% and Disk Util. 3% |
I/O Count Statistics |
1905 I/Os with 52.17 % Reads |
I/O Size Statistics |
63.72 MB with 79.93 % Reads |
Flush Count |
94 |
Test Case File: ImageEditing.IOPattern.txt
5.4.1.3 Video Download
This scenario involves downloading a video over the internet. The observed statistics are as shown in the following table.
Trace Duration |
194 s |
System Utilization Statistics |
CPU Util. 9% and Disk Util. 1% |
I/O Count Statistics |
679 I/Os with 28.42 % Reads |
I/O Size Statistics |
118.17 MB with 41.21 % Reads |
Flush Count |
44 |
Test Case File: VideoDownload.IOPattern.txt
5.4.1.4 Application Install
This scenario involves installing an application over the internet. The observed statistics are as shown in the following table.
Trace Duration |
184 s |
System Utilization Statistics |
CPU Util. 22% and Disk Util. 12% |
I/O Count Statistics |
8566 I/Os with 54.1 % Reads |
I/O Size Statistics |
300.42 MB with 73.12 % Reads |
Flush Count |
390 |
Test Case File: AppInstall.IOPattern.txt
5.4.1.5 Ripping an Audio CD
This scenario involves ripping an audio CD and saving the tracks to disk. The observed statistics are as shown in the following table.
Trace Duration |
179 s |
System Utilization Statistics |
CPU Util. 52% and Disk Util. 1% |
I/O Count Statistics |
2777 I/Os with 22.61 % Reads |
I/O Size Statistics |
97.35 MB with 8.37 % Reads |
Flush Count |
222 |
Test Case File: RipCD.IOPattern.txt
5.4.2 Methodology
In this section, we outline our methodology for running these tests as well as what we measure to assess system performance.
5.4.2.1 Emulating Real-world Workloads
We use a workload emulation methodology to run these tests; this approach reduces the overhead both from a scenario maintainability viewpoint as well as the task of running these tests. Moreover, this technique allows one to stress and focus on the workload aspect of interest, namely disk I/O. The technique essentially involves using tracing to identify the workload details. We then use this detailed workload information to replay the trace.
The workload emulator is essentially an IO replay tool and issues the read, write and flush commands in the order and while adhering to the inter-arrival times observed in the trace. A large file is created on the disk and all I/Os observed in the trace are issued to this file. Since, we are only interested in I/O emulation the CPU utilization as observed above is not reciprocated. Any background I/O, operating system related or otherwise, would result in slightly different I/O statistics during workload emulation. Finally, note that the disk utilization observed can vary significantly from that observed above depending on the disk characteristics.
5.4.2.2 Measuring performance
As before, we seek to compare the system performance when the non-volatile cache feature is enabled to when it is disabled. The performance is measured in the context of the scenario.
- While running these tests we use hooks in the workload emulator to measure the workload metrics of interest. We measure the response time for each I/O operation for each I/O type namely, disk read, disk write and disk flush. Each scenario is run multiple times for both the case when the non-volatile cache feature is enabled as well as when it is disabled. The goal is to ensure that the performance for the two cases is statistically similar without any visible outliers. The performance metrics of interest are the average and maximum I/O time observed for each I/O type. The test fails if any of the following is true:
- Average I/O time for any I/O type for NVC enabled is 10ms longer than the NVC disabled case
- Maximum I/O time for any I/O type for NVC enabled is 20ms longer than the NVC disabled case
6 Appendix: Partner Q&A
Q. Is it okay to re-order non-FUA writes around FUA writes, in any manner between two flushes? Note, that this question only makes sense when write-buffering is enabled.
A. From a disk consistency point only non-FUA writes that can occur and be observed in a bus-trace before a FUA-write between two flushes are the ones that were indeed issued before the FUA-write.
Q. Is it okay to disable write-buffering by default?
A. We take write-buffering to mean the disk setting that is exposed to an end-user, and which historically one can toggle on demand. Please see Section 3.4.6.2 for more details.
Q. The performance requirements around power transitions seem to indicate that it is acceptable to observe performance degradation in a separated-cache solution?
A. In general, one would expect a separated-cache solution to yield a better experience for the end-user around power-transition scenarios such as resuming from standby/hibernate or when booting the system. However, the tests outlined in this document are not comprehensive performance tests per se and merely aim at ensuring that the end-user experience is comparable to what he would observe if using a regular hard disk.
Q. Why are percentage metrics used when specifying the goals for the standby/hibernate performance tests and an absolute metric being used for shutdown?
A. While, a percentage based performance metric is ideal as it is oblivious of the absolute performance of the system being tested. A raw time based performance, however, was necessary for shutdown as Windows has a timeout associated with shutdown (barring a service having registered for the SERVICE_CONTROL_PRESHUTDOWN notification).
Q. Why have representative performance numbers not been provided for the power transition (standby/hibernate/shutdown) performance tests?
A. The performance numbers observed can vary significantly from system to system depending on the hardware configuration. This is also why the performance goals specified are not absolute and have been specified relative to the baseline performance on the same system when the NVCache mode is disabled.
Q. How do you ensure that writes issued for a test are observed in the bus-analyzer trace given that the disk may be spun-down during the test?
A. To ensure that the writes issued during the test for the synthetic data reliability tests are observed in the bus-analyzer trace, each Flush command will result in an automatic issue of an NVCache flush command in succession. Note, that since these tests only aim to verify ordering and do not make any timing measurements, any performance impact is ignored.
Q. Is it okay to invalidate non-volatile cache contents and do a roll-back in the rare case of a double power-failure?
A. The final disk state in the case of a power loss should mimic in essence what is observed on a regular disk. When write buffering is enabled a successful flush on a regular disk implies that the data has been committed to disk. So, non-volatile cache contents can be invalidated to the point the host did not receive a successful flush command for any of the data that gets invalidated. If write-buffering is disabled on the system, none of the writes that were successfully completed can be invalidated.
Q: Why don't H-HDD vendors have to run the evaluation tests?
A: Every premium storage solution is unique and has its own strengths and weaknesses. We evaluate devices on their own merit rather than comparing premium storage solutions. This allows us to focus on the characteristics most appropriate to a given device.
This is potentially a very common scenario in Vista where the system is in the "Balanced" power mode by default. In the default settings for this power mode the non-volatile cache feature is enabled on battery and is disabled when on AC power.
Note that specifying flag FILE_FLAG_WRITE_THROUGH when opening a file implies the file is opened in the write-back mode. Specifying the flag implies the file is opened in the write-through mode.
Run Time: Manual Test: 2 days, Logo test: 5 minutes
Log File: Manual tests: Results.xml, DTM Test: SCSEvaluationResultsChecker.wtl
System Restart Required: No
Test Category:
Supported operating systems for Logo or Signature testing:
Windows Vista
Windows Server 2008
Program:
Requirements
Software Requirements
The test tool requires the following software:
Supported operating system (see list above)
Software components included with the device that is being tested
Hardware Requirements
The test tool requires the following hardware:
Device to be tested
Computer that meets the minimum software requirements
Windows keyboard
Two-button pointing device
Color display monitor capable of at least 1024 by 768 resolution, 32-bits per pixel, 60 Hz
Hard drive with a minimum of 20 GB available on partition C:
Bus analyzer for controller interface type (SATA, ATA, and so on).
If bus analyzer requires a host machine to capture traces, a second machine will be needed to perform this function. This machine need not have the test hardware or meet any test requirements. Its sole purpose will be to capture a bus trace.
Running Separated Cache Solution Evaluation test
Launch the "Separated Cache Solution Evaluation Test (LOGO)" from device console. When prompted for the results file, browse to the file that was created by running the manual test steps previously. Click Load. The test should copy this file and execute on the client.
Code Tour
File Manifest
| File | Location |
|---|---|
SCSEvaluationResultsChecker.exe |
[WTT\TestBinRoot]\nttest\driverstest\storage\wdk\SCSEvaluation |
SCSEvaluationResults.dll |
[WTT\TestBinRoot]\nttest\driverstest\storage\wdk\SCSEvaluation |
SCSEvaluation.msi |
[WTT\TestBinRoot]\nttest\driverstest\storage\wdk\SCSEvaluation |
Test Assertions
5.2.2.1.io.txt Basic Write / Flush Test, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.2.ff.io.txt Basic Write / Flush Test, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.2.io.txt Basic Write / Flush Test, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.3.1.ff.io.txt Basic Write / Flush Test with crash, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.3.1.io.txt Basic Write / Flush Test with crash, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.3.2.ff.io.txt Basic Write / Flush Test with crash, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.3.2.io.txt Basic Write / Flush Test with crash, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.4.1.ff.io.txtBasic Write / Flush Test with power loss, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.4.1.io.txt Basic Write / Flush Test with power loss, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.4.2.ff.io.txt Basic Write / Flush Test with power loss, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.4.2.io.txt Basic Write / Flush Test with power loss, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.5.1.ff.io.txt Basic Write / Flush Test with double power loss, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.5.1.io.txt Basic Write / Flush Test with double power loss, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.5.2.ff.io.txt Basic Write / Flush Test with double power loss, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.2.5.2.io.txt Basic Write / Flush Test with double power loss, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.1.1.io.txt Standby / Resume test, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.1.2.io.txt Standby / Resume test, Write buffering enabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.2.1.ff.io.txt Standby / Resume test with crash during resume, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.2.1.io.txt Standby / Resume test with crash during resume, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.2.2.ff.io.txt Standby / Resume test with crash during resume, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.2.2.io.txt Standby / Resume test with crash during resume, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.3.1.io.txt Standby / Resume test with crash during standby, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.3.2.ff.io.txt Standby / Resume test with crash during standby, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.3.2.io.txt Standby / Resume test with crash during standby, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.4.1.ff.io.txt Standby / Resume test with power loss on resume, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.4.1.io.txt Standby / Resume test with power loss on resume, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.4.2.ff.io.txt Standby / Resume test with power loss on resume, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.4.2.io.txt Standby / Resume test with power loss on resume, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.5.1.io.txt Standby / Resume test with power loss on resume and recovery, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.5.2.ff.io.txt Standby / Resume test with power loss on resume and recovery, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.3.5.2.io.txt Standby / Resume test with power loss on resume and recovery, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.1.1.io.txt Hibernate / Resume test, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.1.2.io.txt Hibernate / Resume test, Write buffering enabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.2.1.io.txt Hibernate / Resume test with crash during resume, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.2.2.ff.io.txt Hibernate / Resume test with crash during resume, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.2.2.io.txt Hibernate / Resume test with crash during resume, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.3.1.io.txt Hibernate / Resume test with crash during hibernate, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.3.2.ff.io.txt Hibernate / Resume test with crash during hibernate, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.3.2.io.txt Hibernate / Resume test with crash during hibernate, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.4.1.io.txt Hibernate / Resume test with power loss on resume, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.4.2.ff.io.txt Hibernate / Resume test with power loss on resume, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.4.2.io.txt Hibernate / Resume test with power loss on resume, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.5.1.io.txt Hibernate / Resume test with power loss on resume and recovery, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.5.2.ff.io.txt Hibernate / Resume test with power loss on resume and recovery, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.4.5.2.io.txt Hibernate / Resume test with power loss on resume and recovery, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.1.1.io.txt Shutdown / Restart test, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.1.2.io.txt Shutdown / Restart test, Write buffering enabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.2.1.io.txt Shutdown / Restart test with crash during shutdown, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.2.2.ff.io.txt Shutdown / Restart test with crash during shutdown, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.2.2.io.txt Shutdown / Restart test with crash during shutdown, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.3.1.io.txt Shutdown / Restart test with power loss during shutdown, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.3.2.ff.io.txt Shutdown / Restart test with power loss during shutdown, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.3.2.io.txt Shutdown / Restart test with power loss during shutdown, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.4.1.io.txtShutdown / Restart test with power loss during shutdown and recovery, Write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.4.2.ff.io.txt Shutdown / Restart test with power loss during shutdown and recovery, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.2.5.4.2.io.txt Shutdown / Restart test with power loss during shutdown and recovery, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.1.1.WC0.io.txt Suspend with heavy write activity, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.1.1.WC1.io.txtSuspend with heavy write activity, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.1.2.WC0.io.txt Suspend with heavy write activity, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.1.2.WC1.io.txt Suspend with heavy write activity, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.2.1.WC0.io.txt Hibernate with heavy write activity, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.2.1.WC1.io.txt Hibernate with heavy write activity, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.2.2.WC0.io.txt Hibernate with heavy write activity, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.2.2.WC1.io.txt Hibernate with heavy write activity, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.3.1.WC0.io.txt Shutdown with heavy write activity, Write buffering disabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.3.1.WC1.io.txt Shutdown with heavy write activity, Write buffering enabled, 1 flush / 10 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.3.2.WC0.io.txt Shutdown with heavy write activity, Write buffering disabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.3.2.WC1.io.txtShutdown with heavy write activity, Write buffering enabled, 1 flush / 2 writes
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.4.1.WC0.io.txt Heavy random write activity with read, write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.4.1.WC1.io.txt Heavy random write activity with read, write buffering enabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.4.2.WC0.io.txt Heavy random write activity, write buffering disabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
5.3.4.2.WC1.io.txt Heavy random write activity, write buffering enabled
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
AppInstall.IOPattern.txt Application installation
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
DocumentDownload.IOPattern.txt Document download
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
ImageEditing.IOPattern.txt Image editing
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
RipCD.IOPattern.txt Ripping an audio CD
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
VideoDownload.IOPattern.txt Video download
Test Assertion GUID:
Windows Logo Program: STORAGE-0012
Additional Information:
Build date: 9/14/2012