Usage of NativeWindow.AssignHandle/ReleaseHandle when unmanaged code is involved.
If an application is Subclassing from managed and unmanaged code, it should not use the NativeWindow.AssignHandle/ReleaseHandle to subclass/unsubclass due to following reasons,
- In a managed application, if we call NativeWindow.AssignHandle to subclass, while that Winproc is still in the subclass chain and some native component calls SetWindowLong or SetWindowSubclass, and then when we call NativeWindow.ReleaseHandle, the call will replace the winproc with User32!DefWindowProc which is not desired in most of the cases and will cause the applicatoni crash most of the time.
- In the above scenario after call to NativeWindow ReleaseHandle, all other WinProc are removed from the chain and User32!DefWindowProc will be in place.
The above problem cannot be potentially solved or may be very difficult to solve in .net due to following reasons.
1. Subclassing creates a linked list of subclasses but Windows doesn’t provide its own mechanism for manipulating that list. We need to be able to have multiple NativeWindows subclassing in the same window and need to be able to detach them in any order.
2. Managed code can finalize and sometimes we have no window procedure we can set back because the procedure is managed code and can also be potentially finalized. When we cannot find a valid window procedure to set as the next window procedure we resort to user!DefWindowProc.
More details on how managed code looks at native WinProc and subclassed procs.
Let’s consider the following example,
- An original window procedure is an application’s main window procedure and has a value of 0x12345678
- Now the nativeWindow subclasses and sets window procedure to 0x11114444 and then stores 0x12345678 as the default window procedure
- Finally, the unmanaged code subclasses, sets window procedure to 0x11112222 and stores 0x11114444 as next in the subclass chain.
At this point, the subclass chain looks like this:
0x11112222 (via SetWindowSubclass) à 0x11114444 (via NativeWindow.AssignHandle) à 0x12345678 (Application’s wndproc)
Now NativeWindow unsubclasses and sees 0x11112222 as the current window procedure.
At this point, NativeWindow only knows that it cannot put 0x12345678 back into the window procedure because someone else has subclassed. So what should it store in the window procedure? It has to put something there or else it will crash. But it doesn’t know that 0x12345678 is the original window procedure. It could be that 0x12345678 was a subclass that was later removed. It always has to assume the worst case which is - 0x11112222 and 0x12345678 are both subclasses that may ultimately include the native window’s window procedure of 0x11114444 in their subclass chain. Therefore it is unsafe for NativeWindow to use either of those values. It’s only recourse is to use a known native window procedure with User32!DefWindowProc.
The Best practice and apotential solution to the above mentioned problem.
- P-invoke to Comctl32.dll ‘s SetWindowSubclass and RemoveWindowSubclass.
- A working sample for the above step is attached. ( Sample is to depict what is discussed and not to be copied to production code - its far from that. )
Bottom Line : NativeWindow.AssignHandle/ReleaseHandle is obsolete and dont use it for any practical purposes.