Narrowing down odd behavior in the Program Compatibility Assistant on Windows Vista
Recently, a fellow Microsoft employee found this blog post that I wrote a while back, and he contacted me with a question regarding some odd behavior he was seeing with the Program Compatibility Assistant (PCA) on Windows Vista. He found that a specific EXE consistently triggered a PCA dialog when he closed it on his system, but nobody else on his team saw this behavior for this EXE on their systems. What made this even more strange was that this EXE included an embedded UAC manifest, which should prevent Windows Vista from triggering any PCA dialogs for the EXE.
After consulting with the Windows application compatibility team, we finally found the root cause and were able to fix this system so it no longer triggers PCA dialogs for this EXE. In the process, we learned some interesting details about how Windows Vista works behind the scenes when deciding whether or not to trigger a PCA dialog. I haven't seen this information documented very thoroughly so far, so I wanted to post about this experience here in case it is helpful to anyone else.
Problem 1 - The embedded UAC manifest was not recognized
The EXE in question had an embedded UAC manifest, but it was embedded using resource ID value 0. Once the EXE was rebuilt with the UAC manifest embedded using resource ID value 1, then Windows Vista was able to find the manifest, and it stopped triggering PCA dialogs on this system.
For reference, the instructions in this MSDN topic can be used to embed a UAC manifest in an EXE using Visual Studio or via command line using mt.exe from the Windows SDK. The examples in this topic demonstrate how to embed the manifest using resource ID value 1, but the topic does not specifically state that you have to use resource ID value 1. The experience we had debugging this issue suggests otherwise.
Problem 2 - Windows Vista logic for triggering PCA was affected by an uninstalled application
Due to the unrecognized UAC manifest described in problem 1, this EXE was treated by Windows Vista as a legacy application, which caused it to be eligible for PCA dialogs when the user exits the EXE. However, PCA is designed to help resolve problems with failing legacy setup programs, and this EXE is not a setup. That left us with another issue we needed to diagnose in order to fully understand the behavior on this system - why was this non-setup EXE being treated as a failing setup?
After discussing this scenario with the Windows application compatibility team, we discovered that Windows Vista monitors events it receives from the UAC virtualization system driver. If Windows Vista finds from these events that an application attempted to create a folder under the Program Files directory and then attempted to create/copy files into a folder under the Program Files directory and those operations fail, then it will treat the application as a failed installer and trigger a PCA dialog if it does not contain an embedded UAC manifest.
Fortunately, the UAC virtualizaton system driver logs events in the event log that can be used to help narrow down the root cause of this type of issue. In the case of this system and this EXE, several events were logged from the Microsoft-Windows-UAC-FileVirtualization source. These events indicated attempts to perform operations on a path under Program Files for an application that has recently been uninstalled.
It turned out that the recently uninstalled application left behind some information in the PATH environment variable and in the registry that caused other programs to attempt to read and write from its installed location under Program Files.
In the case of this EXE, because of these orphaned settings as well as the unrecognized UAC manifest, Windows Vista treated the EXE as a failing legacy setup program and triggered a PCA dialog each time it exited on this specific system. Other users did not see this behavior because they did not have this other application installed and uninstalled on their systems.
Summary of what we learned from this scenario
We were able to fully resolve this issue by doing the following:
- Updating the EXE to embed the UAC manifest using resource ID value 1
- Removing the left over information in the PATH environment variable and in the registry on this system
We also learned a few key things about UAC and PCA on Windows Vista:
- Embedded UAC manifests must be placed at resource ID value 1 in order to be correctly recognized by Windows Vista
- Windows Vista treats programs that attempt to create folders and files under Program Files as setup programs
- Windows Vista logs some events in the event log from the Microsoft-Windows-UAC-FileVirtualization source that can be used to diagnose why a PCA dialog is being triggered for a given EXE
Comments
Anonymous
October 15, 2007
PingBack from http://msdnrss.thecoderblogs.com/2007/10/16/narrowing-down-odd-behavior-in-the-program-compatibility-assistant-on-windows-vista/Anonymous
October 15, 2007
PingBack from http://www.windows-vista.luiscorreia.com/narrowing-down-odd-behavior-in-the-program-compatibility-assistant/Anonymous
October 16, 2007
Great information - this helps avoid mistakes with non-legacy exes. Do you know of any programmatic or manual ways to temporarily suppress the PCA dialog for true legacay cases e.g., running a .NET 1.1 "dotnetfx.exe /?" (the scenario sounds identical to your previous post http://blogs.msdn.com/astebner/archive/2007/05/17/2705372.aspx only with non-beta software)?Anonymous
October 16, 2007
Hi Murman - Unfortunately, there does not appear to be a way to suppress or prevent the PCA logic from kicking in and displaying this dialog when exiting a legacy setup. I've seen similar behavior when running setup packages with the /? argument to see usage, and it can be pretty annoying.Anonymous
October 16, 2007
This would be a really good candidate for an ICE. I know InstallShield has vendor provided Vista ICE's including checking signed/manifested EXE's.... I wonder if it checks the Resource ID when it does it.Anonymous
October 17, 2007
Aaron - thanks. It seemed prudent to settle on an "if it hurts, don't do it" response, and I guess we'll just stick with that strategy. Christopher - the ICE implicitly verifies the resource ID by only checking a manifest in resource ID value 1. It won't give a helpful "you seem to have a useless manifest in ID 0" error, though.Anonymous
October 17, 2007
I think a warning like the above would be useful. I've only done a couple installs that were targeted for logo cert and to be truthful, I just got the binaries handed to me by an external build team and I was only responsible for running validation and providing feedback. It took a quite few iterations as the developers got used to how to implement the new requirements in the source/build and more debug information would have greatly sped that up.Anonymous
October 26, 2007
PingBack from http://systemofadown.wpbloggers.com/?p=139