“Everybody Lies”

Today we present EricLaw’s 2nd law of Software: “If your software platform is sufficiently popular, and it offers a GetVersion API, that API probably lies. ”

Recently, a user of Telerik’s automated web testing product (Test Studio) filed a bug noting that they’d recently upgraded their machines to IE11, but the test tool’s GUI claimed that agents would be running IE9. Looking closer, the test agent actually indicated that it would use a non-existent version, IE 9.11.

How did that happen?

The agent client software checked the machine’s registry to determine the current IE version. Following common and decades-old practice, the code simply looked at the Version subkey inside HKLM\Software\Microsoft\Internet Explorer. If you open RegEdit on a Windows 8.1 machine, the problem is immediately obvious:

As you can see, the Version key contains a bogus IE version: 9.11.9600.16476. On a Windows 8 machine running IE10, the Version key contains 9.10.9200.16384.

So, the Version registry key lies. But why?

The Version key has lied since IE10 because there’s a common and decades-old practice of looking in the registry to retrieve the IE version. Unfortunately, much of the code that does so is inside product installers that refuse to proceed if a minimal target IE version (e.g. 6+) isn’t found. And that crusty old code never imagined there would one day exist a world in which IE’s major version was more than a single digit long. As a consequence, when IE10+ is installed, many products would refuse to install and users would receive bizarre and inaccurate error messages (“Internet Explorer 6 is required. This system only has Internet Explorer 1. ”).

After examining the broad scope of the compatibility problem, the team decided to change the registry format such that the “true” IE version would be preceded by a leading “9.” to accommodate buggy version checkers. This approach generally worked well, but will confuse any software, like Test Studio, that was written prior to the version lie being introduced.

This is only one of many version lies.

User-Agent Lies

The most notorious of the version lies can be found in the browser’s user-agent string. Consider the following simplistic application which hosts the Internet Explorer Web Browser control (WebOC):

The browser first claims to be “Mozilla/4.0”, a lie carried over from the mid-1990s when Netscape Navigator had overwhelming marketshare and every other browser needed to mimic it in order to have a chance of rendering important sites. Today, virtually all browsers claim to be a variant of Mozilla, although most modern browsers (Firefox, Chrome, IE11, etc) purport to be  “Mozilla/5.0”. By default, WebOC hosts run in IE7 Emulation mode, which is why you see both  “Mozilla/4.0” and “MSIE 7.0” in the string; the Trident/7.0 token reveals the control’s true nature. Only if the FEATURE_BROWSER_EMULATION feature control key is set for the WebOC’s host executable will the control default to a later emulation mode. Unlike IE itself, WebOCs do not use the downloaded Microsoft Compatibility View list (which can offer site-specific UA string lies for compatibility).

The  “Windows NT 6.2” token is another interesting one—it lies in numerous ways. First, Windows hasn’t used the “NT” moniker since Windows 2000 was released. Next, my application is running on Windows 8.1, and yet the UA string claims that Windows is v6.2. That’s a lie stacked on top of a lie. With Windows Vista’s release, the Major version returned by GetVersionEx has been frozen at 6; Windows Vista was 6.0, Windows 7 was 6.1, Windows 8 was 6.2, and Windows 8.1 was 6.3. So, even considering the 6.x lie, you’d expect that my test application would be showing 6.3, but it’s not.

That’s because GetVersionEx now directly lies to applications that call it when run on Windows 8.1; it only returns the 6.3 value if the application specifically indicates compatibility with 8.1 inside its manifest. To prevent applications from lying (and, say, claiming compatibility with Windows 9 before it exists), the manifest requires that callers list their supported operating systems using a GUID; each OS version’s GUID is only documented when the new OS version is reaches its public preview stages.

Windows’ AppVerifier tool has long offered developers a means to test whether software is inappropriately limiting itself based on the Windows version; when the HighVersionLie test is enabled, the application being verified will receive a high (and fake) version number when calling GetVersionEx; in this case, Windows claims to be version 8.5.10600:

Unfortunately, not enough applications test with AppVerifier and many still inappropriately perform version checks. In Internet Explorer, we even found that some websites would fail; when Windows Vista was in development, for instance, the TurboTax website refused to load because it didn’t like the Windows NT 6.0 version token.

API Lies

Many other versioning APIs will also return bogus results for compatibility.

For instance, when updating Fiddler to support the new SmartWPAD feature introduced in Internet Explorer 8, I figured I’d ask WinINET for its version information before using IE8’s new INTERNET_PER_CONN_FLAGS_UI flag (since this flag isn’t supported by earlier versions of WinINET).

For this task, I expected I’d use simply call InternetQueryOption(INTERNET_OPTION_VERSION) and look at the resulting version. Unfortunately, as I soon discovered, this call’s return value was frozen at 1.2 in the mid-1990s. In talking to the developers, I learned that this value was also frozen due to compatibility concerns, where applications refused to run on later versions.

To reliably use the INTERNET_PER_CONN_FLAGS_UI flag, you must simply check the return value of the InternetQueryOption call, and if it returns ERROR_INVALID_PARAMETER, retry the call without the flag.

Random Lies and Failures to Lie

You need only peruse this blog to find other cases where version information either already lies, or should do a better job of lying:

• The SSL/TLS version communicated during a HTTPS handshake is a bit of a lie (e.g. TLS1.0 is advertised as SSLv3.1), but some servers still unexpectedly fail when they see a version number that they don’t expect, rather than simply using the latest protocol version they do support.
• For years, the default guidance for Apache would result in disabling keep-alive for any IE version <5 and >9, as I described here.
• The compatMode property is frozen at CSS1.
• …

Other Browsers

Cynical observers everywhere might remark: “No, only Microsoft products lie,” but this is, of course, untrue.

• Opera was the first browser to include a robust system of compatibility lies
  • With its small marketshare, it was forced to emulate many IE proprietary features so it would be detected as a supported browser
  • It was the first major browser to cross the two-digit version boundary; the User-Agent string remains frozen at Opera/9.8
  • It was the first major browser with a “Compat list” that was downloaded to the client with site-specific lies
• Firefox’s User-Agent header contains several lies, including the frozen string Gecko/20100101 
• Firefox’s extension model requires that extensions declare their minimum and maximum supported version; unlike the Windows versioning APIs, the numbers used are entirely predictable and nothing stops extensions declaring compatibility with some future version of the platform. Such lies became a necessity as the Firefox shipping cycle accelerated.
• Chrome’s User-Agent string is designed to spoof Netscape, Firefox, Safari, and KHTML simultaneously. Having said that, Chrome has been generally successful in shipping new versions as such a frenetic pace that any site that attempts to target a specific Chrome version will likely be broken before the development team manages to push their bits to the server.

Everybody lies. It’s usually for good reason, to accommodate someone else’s bad code. Please follow best practices like feature detection, and do your best to write future-proof code.

Thanks!

-Eric

Comments

• Anonymous
  February 26, 2014
  Couldn't the "Mozilla/5.0" part be dropped from the IE user agent string soon? I still don't understand why it's still there, since it doesn't have a meaning anymore. [EricLaw] Sigh...every single blog post about UA strings gets the same comment. No, dropping it would be disastrous. Try it yourself using Fiddler and you'll see why.

• Anonymous
  March 31, 2014
  What we really need is developer re-education to show people how to properly detect version numbers. The amount of apps that tell me i can't install them because my version of windows is too low (on windows 8.1) is astronomical, and comical. surely detecting a two digit number and comparing it to a two digit number are fundamentally simple? You'd think so, right?