The “MSXML Interfaces appearing in my TLB” issue and related topics
Hi all! I have been involved recently in the troubleshooting of a type library-related problem reported by a customer. Since the problem is general enough, I am sharing the basic principles in this post. The topic deals with the way type libraries reference external types, as well as the implications on type library generation and registration.
The Problem
Assume you have a COM interface of your own, defined in an .IDL file, which accepts method arguments of type IXMLDOMXxx. For example, let’s say your interface has a method that looks like this:
[id(1), helpstring("method Load")] HRESULT Load([in] IXMLDOMNode* pXDN);
For your IDL to be compiled successfully by the MIDL compiler, the reference to IXMLDOMNode needs to be resolved. This happens automatically if you use the default import statements that are generated for a new IDL file in Visual Studio:
import "oaidl.idl";
import "ocidl.idl";
So your .IDL file compiles successfully and everything seems to be OK.
Now let’s use OLEVIEW, a utility program that ships with Visual Studio and the Platform SDK, to look at the generated type library (.TLB file). If you open the type library, you’ll see that IXMLDOMNode, along with many other IXMLDOMXxx interfaces, are included in the generated .TLB file:
This problem is more common than one may initially think, as shown by the many posts like this one out on the net:
https://www.eggheadcafe.com/software/aspnet/30162765/msxml-interfaces-appearin.aspx
We will be using Visual Studio 2008 and the Platform SDK version 6.0A. I would expect, but I cannot guarantee, that the following analysis is not version-dependent.
Type Library Registration
You may wonder at this stage: why is it so relevant to reference IXMLDOMNode from the generated TLB, instead of embedding it?
The main reason is that, when the TLB is registered, the interface-related registry keys under HKCR\Interface are changed, so as to associate IXMLDOMNode with your own type library.
Note
As an aside, note that the registration of a type library can happen in different ways. The API RegisterTypeLib does the registration, but most often a type library is registered as part of a COM dll registration through regsvr32.exe, or by calling its DllRegisterServer exported function. Yet another way to register a type library is by calling regsvcs or regasm on a .NET Assembly which has COMVisible types. In all these cases, the type library can be in a .TLB file or embedded as a resource in a .DLL.
Back to the point: the registration of the type library modifies keys under HKCR\Interface, we said. In the case above, if we look at the registry key {2933BF80-7B36-11d2-B20E-00C04F983E60}, the Interface ID (IID) of IXMLDOMNode, before we register our type library:
and after:
we understand what the difference is. Note that F31AF329-0FE0-4640-80F8-40D187FF4D8B is the Library ID (LibID) of the generated type library.
The fact that now the XML interfaces are associated with your type library instead of the standard type library included in msxml6.dll is not going to create trouble, initially. What happens, however, if you remove your type library, typically as part of uninstalling a program? The reference, in the registry, to your type library won’t be resolved and runtime errors will occur.
Changes in Vista and later OSes
Actually, when speaking about registering a type library, the outcome is different depending on the operating system and the way registration occurs. Let’s go deep into this as it provides an interesting view on some nifty details.
In operating systems up to, and including, Windows 2003, the registry key HKCR\Interface has the permission “Full Control” for the Administrators group:
Also, the “Full Control” permission is inherited by subkeys, so any user in the local Administrators group can do the registration and successfully update the keys.
From Vista, Administrators have “Read” permission on the registry keys representing LibIDs, and only TrustedInstaller has “Full Control” permission by default:
This means that, even if you are doing the registration with elevated privileges, or with UAC disabled, those registry keys cannot be changed – only setup programs, by having the TrustedInstaller credentials, will be able to do the registration.
So what happens if you do the registration through RegisterTypeLib, or by running regsvr32.exe from the command prompt, or if you call DllRegisterServer directly, in Vista and later OSes? Turns out the outcome is fairly interesting.
- RegisterTypeLib() fails with error 0x8002801c (TYPE_E_REGISTRYACCESS: “Error accessing the OLE registry”).
- DllRegisterServer(): this really depends on how DllRegisterServer() is implemented in the DLL that includes the type library. The default ATL implementation returns the HRESULT of RegisterTypeLib unchanged, so it also returns TYPE_E_REGISTRYACCESS. Other implementations may turn the error into a more generic HRESULT like E_FAIL.
- regsvr32.exe succeeds and shows the message box “DllRegisterServer in D:\ATLDll\Release\atldll.dll succeeded”. Uh? How can it be, if all it does is to call DllRegisterServer? First of all, we can double-check that the registry keys have not been changed. So what happens is that the call to DllRegisterServer failed, but nonetheless regsvr32.exe reported a successful return. This “magic” happens by means of the Shim Technology: if the hosting process is regsvr32.exe, the return value of the RegSetValueEx() API is changed through a shim.
The logic behind the tightening of security permissions on these registry keys should be fairly clear: if any registration of a COM Dll is allowed to overwrite the registry keys related to a system TLB like msxml6.dll (Windows 2003 behavior), the system may misbehave as soon as this Dll is removed. This is one of those scenarios where re-registering msxml6.dll (or another system dll) mysteriously fixed the problem until Windows 2003.
The logic behind the shim in regsvr32.exe is more subtle and has to do with compatibility. Let’s assume we re-register a dll like msxml6.dll by running regsvr32.exe. Since the registration tries to write values to those protected registry keys, the registration would fail. Keep in mind, however, that those registry keys already have those values from the registration that occurred at installation time, when a principal with the TrustedInstaller credentials wrote them. Reporting a failure from regsvr32 would make the user think that something went wrong, which is not the case here. The same behavior takes place if we are registering a custom TLB that happens to embed the definition of the IXMLDOMXxx interfaces.
All right, we came to know the details of type library registration and the “magic” of shims, so it is time to go back to our main topic.
import and importlib
There are essentially 2 ways you can make a type visible within a type library: import and importlib.
import really imports the types in the current .IDL and, as a consequence, in the generated type library (in other words, it does what its name advertises ).
importlib reads the types from a type library and makes them available for type resolution. If type resolution happens successfully, the generated type library will contain a reference to the original type defined in the external type library. Note that importlib can only be used inside the library block of an IDL file.
What happens if a type is referenced by both import and importlib? The MIDL compiler resolves agains the first type, which means the type imported through the import statement (importlib can be used within the library block, whereas import is typically used at the beginning of the IDL file).
With reference to the problem at hand, if we use both “import "ocidl.idl" and importlib(“msxml6.dll”), the generated type library will still embed the IXMLDOMXxx interfaces. This information, however, gives us an easy solution to our problem: use importlib(“msxml6.dll”) without import(“ocidl.idl”). So the following IDL file:
import "oaidl.idl";
//import "ocidl.idl";
[
uuid(F31AF329-0FE0-4640-80F8-40D187FF4D8B),
version(1.0),
helpstring("MyItfLib 1.0 Type Library")
]
library MyItfLib
{
importlib("stdole32.tlb");
importlib("stdole2.tlb");
importlib("msxml6.dll");
[
object,
uuid(BBA99D28-318E-4d69-A835-822EBDCEDA29),
oleautomation,
helpstring("IMyItf Interface"),
pointer_default(unique)
]
interface IMyItf : IUnknown
{
[id(1), helpstring("method Load")] HRESULT Load([in] IXmlDOMNode* querySettingsNode);
};
};
will generate a type library with only the IMyItf interface defined. IXMLDOMNode is referenced from the external TLB:
Unfortunately, this is not the end of the story . There are special cases (not so special, actually) where this simple approach does not work. Read on…
Forward Declarations and Type Dependencies
Using import without importlib in the previous section did the job. This does not work, however, if we define interfaces outside the library block (forward declaration). For example:
import "oaidl.idl";
//import "ocidl.idl";
[
object,
uuid(BBA99D28-318E-4d69-A835-822EBDCEDA29),
oleautomation,
helpstring("IMyItf Interface"),
pointer_default(unique)
]
interface IMyItf : IUnknown
{
[id(1), helpstring("method Load")] HRESULT Load([in] IXmlDOMNode* querySettingsNode);
};
[
uuid(F31AF329-0FE0-4640-80F8-40D187FF4D8B),
version(1.0),
helpstring("MyItfLib 1.0 Type Library")
]
library MyItfLib
{
importlib("stdole32.tlb");
importlib("stdole2.tlb");
importlib("msxml6.dll");
interface IMyItf;
};
in this case, since we are referencing IXMLDOMNode outside the library block, the reference to IXMLDOMNode cannot be resolved and we get the following error:
1>.\MyItf.idl(13) : error MIDL2025 : syntax error : expecting a type specification near "IXmlDOMNode"
1>.\MyItf.idl(13) : error MIDL2026 : cannot recover from earlier syntax errors; aborting compilation
If you are the author of the IDL file, the solution here is pretty easy: avoid the forward declaration and place everything in the library block. Unless you have a complex set of dependencies (for example, interface A uses interface B in some of its methods, and interface B uses interface A in some of its methods), this is doable and effectively addresses the problem.
There is another potential issue though: what if your IDL file needs another type that is defined (either directly or indirectly) in ocidl.idl? If this happens, you need to import ocidl.idl, but by doing that you are also including msxml.idl and hence embedding the IXMLDOMXxx interfaces in your TLB again!!
Here is an example of such an interface (IOleControl is defined directly in ocidl.idl):
[
object,
uuid(BBA99D28-318E-4d69-A835-822EBDCEDA29),
oleautomation,
helpstring("IMyItf Interface"),
pointer_default(unique)
]
interface IMyItf : IUnknown
{
[id(1), helpstring("method Load")] HRESULT Load([in] IXMLDOMNode* querySettingsNode);
[id(2), helpstring("method InitControl")] HRESULT InitControl([in] IOleControl* pControl);
};
Ok, this is admittedly the trickiest part, so let’s take a deep breath before we go on .
DO_NO_IMPORTS to the rescue
Some of you may legitimately wonder: why do we need to import ocidl.idl to have the definition of IOleControl? Couldn’t we follow the same steps that we used for IXMLDOMNode and use importlib against the type library that defines IOleControl?
Yes, we could. The problem is that there is no type library in the sytem that defines IOleControl. This may seem strange, but there is a reason (there is a reason for everything, isn’t there? ): type libraries are used mainly for type-library (also known as OleAutomation, or Universal) marshaling. If an interface defines its own marshaling, there is no need for a type library. Here is how the registry looks like for IOleControl:
As you can see, there is no TypeLib subkey. So we really need to import ocidl.idl in order to have the definition of IOleControl.
Let’s have a look at how ocidl.idl imports the IXMLDOMXxx interfaces. At the beginning of ocidl.idl we have:
#ifndef DO_NO_IMPORTS
import "oleidl.idl";
import "oaidl.idl";
import "servprov.idl";
import "urlmon.idl";
#endif
and near the beginning of urlmon.idl we have:
import "objidl.idl";
import "oleidl.idl";
import "servprov.idl";
import "msxml.idl";
msxml.idl defines the IXMLDOMXxx interfaces. So it seems that we may be able to achieve our goal by using the preprocessor definition DO_NO_IMPORTS. If we do that (best way is to specify it in the “Preprocessor Definitions” test box in the .IDL file properties, equivalent to the /D command-line option, instead of using #define in the .IDL file), however, we get the following output from the MIDL compiler:
1>C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\oaidl.idl(64) : error MIDL2025 : syntax error : expecting a type specification near "ULONG"
1>C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\oaidl.idl(65) : error MIDL2026 : cannot recover from earlier syntax errors; aborting compilation
Note that the errors are generating while compiling oaidl.idl, which is imported before ocidl.idl. The reason is clear if we look near the beginning of oaidl.idl:
#ifndef DO_NO_IMPORTS
import "objidl.idl";
#endif
Since objidl.idl defines some basic types like ULONG, we need to include it explicitly before objidl.idl. Once we do this, we still get errors because of unresolved types. We can fine-tune import of other IDL files until we get to this:
import "wtypes.idl";
import "unknwn.idl";
import "objidl.idl";
import "oaidl.idl";
import "oleidl.idl";
import "ocidl.idl";
With this sequence of imports we still get a MIDL compiler error:
1>C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\ocidl.idl(1830) : error MIDL2025 : syntax error : expecting a type specification near "IBindHost"
1>C:\Program Files\Microsoft SDKs\Windows\v6.0A\include\ocidl.idl(1830) : error MIDL2026 : cannot recover from earlier syntax errors; aborting compilation
IBindHost is defined in urlmon.idl, which is imported conditionally in ocidl.idl (see first code snippet in this section). However, importing urlmon.idl before ocidl.idl is not a solution, because urlmon.idl imports msxml.idl, as shown by this import sequence at the beginning of urlmon.idl:
import "objidl.idl";
import "oleidl.idl";
import "servprov.idl";
import "msxml.idl";
So here we are apparently at a dead end: we would need to import urlmon.idl because ocidl.idl needs the definition of IBindHost, but if we do that we also import msxml.idl, which will cause the embedding of the IXMLDOMXxx interfaces in our type library.
One possible way out here is to create your own custom IDL file, say MyBindHost.idl, which includes IBindHost and its dependencies, and then to import it in your IDL file in place of urlmon.idl:
import "wtypes.idl";
import "unknwn.idl";
import "objidl.idl";
import "oaidl.idl";
import "oleidl.idl";
import "mybindhost.idl";
import "servprov.idl";
import "ocidl.idl";
This works and the IDL file compiles successfully to a TLB which does not include the definition of the IXMLDOMXxx interfaces. The TLB does include the definition of IOleControl, but this is not an issue: since the interface is not declared with the dual or oleautomation attributes, it won’t register itself as the marshaler of IOleControl.
A suitable MyBindHost.idl is attached at the end of this post.
IUnknown is a special case
At this point you may wonder about the interface IUnknown, which seems to have a different behavior: we need IUnknown in our IDL because it is the base interface of our interface:
...
interface IMyItf : IUnknown
{
...
};
IUnknown is defined in unknwn.idl, which we are importing either directly or indirectly. IUnknown is also made available through importlib of stdole32.tlb and stdole2.tlb. In such a case, you would expect that the first definition is matched, and so we should see IUnknown in the generated type library, but this is not the case.
The reason is that the MIDL considers IUnknown (and potentially other interfaces) differently, and therefore a reference to it is generated in these cases, instead of embedding IUnknown in the generated type library.
Summary
Some key points of this post:
- If you are authoring IDL files, you need to understand the difference between referencing a type in the type library you generate, and embedding it.
- It is very important to avoid embedding system types into your own type library. A common case when this may happen is with the IXMLDOMXxx interfaces.
- In Vista and later operating systems, permissions on registry keys prevent the association of a system interface with a different type library. However, type library registration may fail if not done through regsvr32.exe.
- If your solution consists of several type libraries, the same principles apply: you should define a type in one type library only, and reference it from other type libraries by using importlib. If this causes circular references, factor out the involved types into one type library and reference it from the other type libraries.
Comments
- Anonymous
June 03, 2010
Finally a blog post explaining the logic (and illogic) of MIDL. Kudos!