Quality Control for Windows Embedded Standard 2011 Drivers

One of the nice features of Windows Embedded Standard 2011is that we have the same drivers available as Windows 7. That means that we have a wide range of drivers available (500+), and the drivers have gone through extensive testing with the Windows team. For Windows Embedded Standard 2011 we want to give our customers the flexibility to set up their runtimes with the drivers they want. To accomplish this, most of the drivers are packaged separately from the base Windows Embedded Standard 2011image (called eCore), allowing the user to add them to the image as they wish. The base image does come with some drivers, but only drivers necessary for the critical operations of the computer. For testing drivers in Windows Embedded Standard 2011,we can’t duplicate all of Windows team’s testing. We do want to run test cases in all of the main test categories, but mostly we want to focus our testing on the areas where Windows Embedded Standard 2011differs from Windows 7.

For instance, in Windows Embedded Standard 2011the customer can add or omit the features they chose, so the runtime environment can be vastly different than the original Windows 7 environment the driver was built for. An issue arises if the driver has a dependency on a service, file, registry key, etc. that wasn’t added to the runtime. In order to mitigate this problem we’ve analyzed the dependencies for each driver. If a driver has a dependency on a feature not included in the base image, then that information is added to the metadata of the driver packages. That way whenever you select a driver package you will know if any other packages are required to make the driver work. Since the base Windows Embedded Standard 2011 image is designed to be a self contained working unit, the drivers in it don’t have dependencies outside of the image. But we don’t just rely on the dependency analysis; we also do runtime checks for dependency issues. To that end, our testing is predominantly done with the minimum features added. We also leverage the testing done by the feature teams and run various customers scenarios to give us greater coverage.

Another area that differs somewhat from Windows 7 is the installation of the driver packages. While we use many of the same components as the Windows 7 desktop operation system, we build our own packages and often add them to an image in greater numbers. You can install driver packages either through setup or on the runtime. When installing with setup you can select specific drivers to install by using an unattend file, or you can use setup’s driver automapping feature to match drivers to your devices automatically. After a Windows Embedded Standard 2011install, you can add a package either online or when booted into WinPE 2.0 from another drive. Every installation method is tested, but there are also different stages to verify. We have to check the ‘package’ installation, which is basically about verifying where files are placed onto the image. Then there is the second way of installing drivers, in which Windows 7 components within Windows Embedded Standard 2011 match drivers to devices, and then install the drivers. We verify that the driver is running without any errors, and then extend that testing with our feature set testing. Also, we address the fact that customers will add multiple packages at once, so we stress the system with the maximum packages available. Lastly, we test whether the driver package can be updated.

As mentioned previously, we build our own driver packages, so we have to make sure that the payload is correct and the metadata included is correct. If the packages aren’t well formed, that could impact installation, setup’s driver automapping feature, package dependency resolution, or servicing of packages. In each package there is a deployment manifest, an update manifest, the driver installation file, and the files needed by the driver. The files with the metadata in them contain interconnected data, build data, and installation data that has to be verified. We also have build files associated with these packages that help determine things like what packages we have, or what language resources are available. Automation is used to verify all of these factors on a daily basis. Plus, we supplement the package verification with our installation testing mentioned above.

So we’ve mentioned what we look for in the differences between Windows 7 and Windows Embedded Standard 2011, but we also test the more typical test areas. Some of the other categories we test are Ad hoc, HW coverage, Performance, Stress, Application compatibility, Integration testing, and Language testing. We try to leverage the Windows team’s tests whenever possible, but we also build test cases and automation whenever we need to get the appropriate coverage for a test area.

- Brent