Test Automation: Programmatic Platform Profiling
Occasionally, the execution or outcome of a test depends on the operating system version on which the test is executed. Platform profiling is important because subtle differences in operating system platforms can affect how certain tasks are carried out, inclusion or exclusion of specific features or capabilities, work-arounds, or even changes in deterministic or declarative oracles that determines pass, fail, or indeterminate results. For example, accessing certain features on the Vista platform will trigger the User Account Control (UAC) dialog, whereas this feature does not exist on previous versions of Windows.
It is easy for a tester to select the appropriate OS version when executing tests manually and determine the correct steps or settings and determine the correct results of a test based on their cognitive expectations for an outcome, But, in an automated test system where tests are distributed across the wire to various machine configurations and operating system platforms the test designer must programmatically detect the operating system version in automated tests that need to be profiled for the appropriate platform. Programmatic platform profiling negates the need to write several different tests for a single test case that is dependent on variations in the operating system versions. Detecting the operating system version at runtime allows the automation test designer to handle subtle differences between the operating systems within one one test case.
Detecting the major and minor OS versions with C# is relatively straight forward using the System.PlatformID enumeration and the System.Environment.OSVersion property and OperatingSystem class members.
The System.PlatformID enumeration detects the major operating platforms including:
- Win32Windows - versions of Windows 9x, including Windows ME
- Win32NT - versions of Windows 2000, Xp, and Vista
- WinCE - versions of Windows CE
- Unix - versions of the Unix OS
Once the platform is determined, the test designer can drill down further using the Environment.OSVersion.Version.Major and Environment.OSVerions.Version.Minor members to find the specific major and minor operating system version number. If tests are dependent on specific revisions or service packs of the major operating system then the integer or string values can be obtained using the OSVersion.Version.Revision or OSVersion.Version.Build members to determine specific versions of an operating system. Some versions of Windows used alphabetic characters in the revision number, so in those cases the revision should be converted to and compared using a string object as represented by the constant values for Windows 95 retail, Windows 95 OSR 2.1 and Windows 95 OSR 2.5 and Windows 98 Second Edition. Also, the OperatingSystem.ServicePack property can be used to determine if, and what service pack has been installed on the operating system.
// Major NT Kernel Versions
private const int WINDOWS_NT6_KERNEL = 6;
private const int WINDOWS_NT4_KERNEL = 4;
private const int WINDOWS_NT5_KERNEL = 5;
// Minor NT Kernel Versions
private const int WINDOWS_2000 = 0;
private const int WINDOWS_XP = 1;
private const int WINDOWS_XP2003 = 2;
// Service Packs
private const string SP1 = "Service Pack 1";
// Minor Win9x Kernel Versions
private const int WINDOWS_95 = 0;
private const string WINDOWS_95_SP1 = "950A";
private const int WINDOWS_95_OSR = 3;
private const string WINDOWS_95_OSR2X = "950B";
private const string WINDOWS_95_OSR25 = "950C";
private const int WINDOWS_98 = 10;
private const string WINDOWS_98SE = "2222A";
private const int WINDOWS_ME = 90;
private static string GetOSVersion()
{
OperatingSystem osVersionInfo = Environment.OSVersion;
string osVersion = string.Empty;
switch (osVersionInfo.Platform)
{
case PlatformID.Win32NT:
switch (osVersionInfo.Version.Major)
{
// Windows NT 4.0
case WINDOWS_NT4_KERNEL:
osVersion = "NT4";
break;
case WINDOWS_NT5_KERNEL:
switch (osVersionInfo.Version.Minor)
{
// Windows 2000
case WINDOWS_2000:
osVersion = "Win2K";
break;
// Windows Xp
case WINDOWS_XP:
osVersion = "WinXp";
break;
// Windows Xp 2003 Server
case WINDOWS_XP2003:
osVersion = "WinXp2003";
break;
}
break;
// Windows Vista and Server 2008
case WINDOWS_NT6_KERNEL:
if (osVersionInfo.ServicePack == SP1)
{
osVersion = "Vista SP1";
}
else
{
osVersion = "Vista";
}
break;
}
break;
case PlatformID.Win32Windows:
switch (osVersionInfo.Version.Minor)
{
// Windows 95, OSR 1.0, OSR 2.0
case WINDOWS_95:
if (osVersionInfo.Version.Revision.ToString() == WINDOWS_95_SP1)
{
osVersion = "Windows 95 SP1";
}
else if (osVersionInfo.Version.Revision.ToString() == WINDOWS_95_OSR2X)
{
osVersion = "Windows 95 OSR 2.0";
}
else
{
osVersion = "Win95";
}
break;
// Windows 95 OSR 2.1, OSR 2.5
case WINDOWS_95_OSR:
if (osVersionInfo.Version.Revision.ToString() == WINDOWS_95_OSR2X)
{
osVersion = "Win95_OSR2.1";
}
else if (osVersionInfo.Version.Revision.ToString() == WINDOWS_95_OSR25)
{
osVersion = "Win95_OSR2.5";
}
break;
// Windows 98 and Windows 98 Second Edition
case WINDOWS_98:
if (osVersionInfo.Version.Revision.ToString() == WINDOWS_98SE)
{
osVersion = "Win98SE";
}
else
{
osVersion = "Win98";
}
break;
// Windows ME
case WINDOWS_ME:
osVersion = "WinME";
break;
}
break;
}
return osVersion;
}
Currently, C# does not have a property or method to determine the specific edition of an operating system. If a test is dependent on a specific edition of Windows 2000, Windows Xp, or Windows 2003 operating system edition then we need to invoke the Win32 GetVersionEx() function. If a test is dependent on a specific edition of Windows Vista or Windows Server 2008 then we can invoke the Win32 GetProductInfo() function.
Programmatically detecting the operating system version at runtime enables the test designer to design one test that can execute on a different operating system versions by allowing for specific control flow of a test based on the operating system the test is running. Programmatic platform profiling also negates the need to rewrite the same fundamental test over and over again to handle differences in behavior or expectations of operating system versions. It can also reduce some long term costs of test case maintenance, give the test designer greater control over the execution of an automated test, and increases the reusability and the maintainability of a test case that must be ran on multiple operating system versions.
Quite simply, programmatic platform profiling is a best practice in software test automation when a given test is dependent on the profile of the platform on which it is executing!
Comments
Anonymous
July 02, 2008
Hi, Which approach below are you using typically and what do you recommend? [pseudo code] #1 Test code branches strOS = GetOSVersion If strOS = "Win95" Then RunTestWin95 #2 Common test code with OS-dependant run-time specific handling strOS = GetOSVersion RunTest (strOS) Function RunTest (strOS) If strOS = "Win95" Then... End Function I use #2 with our main AUT using GetAUTVersion within a test case but I am curious what is regarded as recommended best practice in this area.Anonymous
July 03, 2008
Hi Stth10, I agree with you that option #2 (common code with OS dependent runtime handling) is a better practice as it would make maintainability of the code much easier and have less redundancy.