Server Testing Overview
Windows Server testing is primarily stress-oriented testing that include client/server I/O, network stress, CPU consumption, and memory consumption. The specific tests you must run depends on the features that you implement on the server.
This section only describes System.Server tests. A complete Windows Server Certification requires several tests that are comprised of System.Client, System.Fundamentals and System.Server tests. Windows Hardware Certification Kit (Windows HCK) Studio detects all features on a server.
General server stress testing
Several kinds of stress tests get run against a server, including basic system functionality, system stress and shutdown/restart tests. LoadGen is a test tool that generates load on a system under test (SUT). LoadGen is started on the master client and can use multiple stress client computers to generate network load on the SUT.
System functionality tests
The system functionality tests are individual tests of the capabilities of the system. Some tests are run for every system, and some tests only run if the capability exists in the system.
System stress test
The System Stress Test consists of several server scenario workloads that operate from the user level address space that is applied to the system to exercise the system hardware, system-specific devices and drivers, network and storage adapters and drivers, and any filter drivers that might be part of the system configuration, such as multipath storage drivers, storage or file system filter drivers, or intermediate layer network drivers.
The workloads applied are
SQL I/O Simulation
Local Storage I/O
Disk Stress with Verification
Client-Server Storage I/O
Winsock Network Traffic
These workloads automatically scale to the number of network and storage adapters in the system that have connected clients or storage devices, respectively. For example, if the test discovers one network adapter and one storage adapter (together with the necessary connected clients or storage devices, respectively), the test creates workload processes for that number of adapters to provide the stress workload. If the system has multiple network and storage adapters, test processes are created for each of those adapters, drivers, and connected resources (clients or storage devices) to provide the same relative stress workload. Additionally, the network and storage adapters and their respective resources do not need to be the same type. For example, Gigabit Ethernet and 10 Gigabit Ethernet adapters can be tested at the same time, as long as network clients are connected to both devices. Similarly, Fibre Channel and iSCSI storage adapters can be tested at the same time, as long as the appropriate storage media is attached. Any HBA units that are attached to the SUT must be connected to the appropriate type of storage media.
The system test achieves the same relative amount of stress on the system, regardless of the number or type of processors, amount of memory, or the number of network and storage adapters in the system. The test detects the number of processors/cores in the system, as well as the amount of memory in the system. The test then creates as many processor-specific and memory-specific stress processes as are needed to achieve a predetermined level of processor and memory utilization and will terminate those processes if the utilization level exceeds the predetermined level of stress. Therefore, the level of utilization for those resources is always commensurate with the capabilities of the system. A system that supports only a few processors/cores and an appropriate amount of memory for the system has the same relative levels of stress as a larger system with more processors/cores and a greater amount of memory.
Shutdown/Restart Test
The server test also includes a shutdown and restart test. This test signals the system to shut down and restart. The test records the event log information related to shutting down and restarting the system, such as vetoes that prevent shutdown, the startup event, and any driver errors that are received after restarting the system. This test makes sure that all device drivers in the system comply with system shutdown, do not veto, and cleanly restart in the system without conflicting with other drivers. For more information, see I/O Completion Cancellation Guidelines.
There are 3 specific tests:
LoadGen Server Stress - Run First - Set Machine Policies (run time < 30 minutes)
LoadGen Server Stress - Run First - Start Test for Server (run time = 24 hours)
LoadGen Server Stress - Run Last - Reset Machine Policies (run time < 30 minutes)
You must schedule the LoadGen Server Stress - Run First - Set Machine Policies" job before you run the "LoadGen Server Stress - Start Test for Server" job. You must schedule the "LoadGen Server Stress - Run Last - Reset Machine Policies" job after the Start Test for Server job has finished.You must schedule the Run First and Run Last jobs only one time for each submission, but you must schedule and run the Start Test job multiple times until it passes. You must also schedule the Run Last - Reset Machine Policies job if you are going to schedule other different jobs in the same machine pool.
Server virtualization validation program (SVVP) testing
Two kinds of virtualization tests are run against a server, including virtual machine functionality tests and SVVP System functionality tests. The system can be a standalone server or a virtual machine. LoadGen is started on the master client and can use multiple stress client computers to generate network load on the system under test.
Virtual machine functionality tests
The functionality tests are individual tests of the capabilities of the product's virtual machine implementation.
SVVP System functionality tests
The SVVP System functionality tests validate the functionality of the following aspects and components of the virtual machine:
Virtual PCI I/O
Virtual SMBIOS
Virtual Timers
Virtual ACPI and PNP functions
The correct operation of the storage capabilities of the virtual machine
The appropriate signing by Microsoft of all the drivers included
Virtualization products correctly expose to the instance of the running operating system, the fact that the operating system is running in a virtual environment.
Additional features testing
Server systems might have additional functionality beyond that which is required for Windows Server Certification. The additional features for which a system can test and qualify are as follows:
These additional feature tests are in the Windows HCK test harness together with the tests that are listed for systems. Vendors whose systems can meet the requirements for these additional features must select and run the required tests. The Fault Tolerant tests exercise and confirm the ability of a fault-tolerant system hardware, devices, and drivers to have a hardware failure and continue to operate without impacting clients that are connected to the server over on the network. The Enhanced Power Management tests validate that the systems supports the CPUID feature flag, processor p-states, and other functionality needed for Windows Serverto manage the power of the system.
For more information, see the "Power Supply, Metering, and Budgeting Interface" section in the ACPI 4.0 or later specification and “Processor Power Management in Windows Vista and Windows Server.
Note that a system can support none, some or all of the above features, such as both enhanced power management. For a vendor to validate that the system meets the requirements for one or more additional features, one or more of the additional feature tests must be selected and run. Those results are then submitted together with the results of the Server Certification tests. The additional feature test results cannot be submitted separately from Server Certification test results.
Run time for these additional feature tests varies, depending on the selected tests. If the Server Certification test is selected only, testing takes approximately two days, with the Loadgen test taking one day to run and the remaining tests using the remaining time.Additionally, if the system being tested includes audio, video or other devices and drivers, those will be exercised and increase test time.
Minimum required server test
The following is a minimum list of tests that you must run on all servers. For minimum test listed under System.Client or System.Fundamentals, you must review the appropriate prerequisite section for each test.
| Required Server Test | Test Category |
|---|---|
ACPI Logo Test |
System.Fundamentals |
Boot Test (SYSTEM) |
System.Fundamentals |
Debug Capability Test (Logo) |
System.Fundamentals |
Disk Stress (SYSTEM) |
System.Server |
Hal timer tests (HCT) |
System.Fundamentals |
NX Test |
System.Fundamentals |
PCI Hardware Compliance Test For Systems |
System.Fundamentals |
Secure Boot Logo Test |
System.Fundamentals |
Secure Boot Manual Logo Test |
System.Fundamentals |
Signed Driver Check (CheckLogo) |
System.Fundamentals |
SMBIOS HCT |
System.Fundamentals |
System - PNP (disable and enable) with IO Before and After (Certification) |
System.Fundamentals |
System - Sleep and PNP (disable and enable) with IO Before and After (Certification) |
System.Fundamentals |
System - Sleep with IO Before and After (Certification) |
System.Fundamentals |
USB Boot test (SYSTEM) |
System.Fundamentals |
USB Exposed Port System Test |
System.Fundamentals |
USB xHCI Register System test |
System.Fundamentals |
USB3 Termination |
System.Fundamentals |
Verify virtualized environment |
System.Server |
Verify processor virtualization support |
System.Server |
WHEAHCT Logo |
System.Server |
Win8 LoadGen Server Stress - Run First - Set Machine Policies |
System.Server |
Win8 LoadGen Server Stress - Run Last - Reset Machine Policies |
System.Server |
Win8 LoadGen Server Stress - Start Test for Server |
System.Server |
Related topics