通过

Nested message loops are evil (if you're a platform)

  • 项目

When you're writing a platform, you sometimes need to make hard choices between making the common cases easy to use and making the corner cases work 100% correct. Nested message loops have offered us many such challenges. Nested message loops -- Dispatcher.PushFrame in WPF -- are really convenient, because you can pop-up modal UI and get your answer without ever unwinding your call stack. But at the same time, they open the door to some really strange situations and weird deeply nested call stacks.

One issue I saw recently was someone called MessageBox.Show() while inside HwndHost.BuildWindowCore. Sounds innocuous enough, but WPF won't let you do it -- tells you "Cannot perform this operation while dispatcher processing is suspended." Why? Well, you guessed the first part -- MessageBox.Show() will push a nested message loop in order to display that little message box UI. So? Well, WPF called Dispatcher.DisableProcessing(), which is another way of saying "no nested message pumps allowed".

Why did we do that? Well, HwndHost doesn't actually care. But BuildWindowCore is called when WPF is in the middle of doing layout -- and layout runs under disable processing. (As of this writing, layout is actually the only time we disable processing) The reason we did that is because, well, reentrant layout is a bug farm, as we learned from our days in Internet Explorer -- and the only way we could figure out how to prevent starting a new layout run while in the middle of layout was to disallow nested message pumps. And better still, the CLR's default locking mechanism actually pumps messages when the lock is under contention. (I never liked that design, but it does solve a number of deadlock issues with STA COM objects)

Our original solution was to avoid the CLR's locks, but we couldn't stop applications from using them. Then we tried, through the magic of SynchronizationContext, to pump messages inside locks a little differently, but it turns out that despite what filters you may pass to GetMessage(), all SendMessage's will go through -- including WM_SIZE, which triggers layout. We also tried a couple variations of ignoring WM_SIZE at various key times, but never found a way that didn't fail under stress testing. So, we finally created Dispatcher.DisableProcessing(), which temporarily changes CLR lock behavior to do no pumping, refuses to let you use WPF to push a nested loop, and gets really angry at you if a message arrives at our wndproc through other means (e.g., you wrote your own message loop).

Fortunately, there was a happy ending to the HwndHost.BuildWindowCore/MessageBox.Show() issue -- don't call MessageBox.Show() in BuildWindowCore(). <g> Allow me to explain -- if you need to pump messages while you're building your hwnd, that's fine -- just make sure to build your hwnd before BuildWindowCore is called, and hang onto the hwnd until BuildWindowCore is called.

Comments

  • Anonymous
    July 09, 2006
    Discusses WinForms interop and how to work around an issue which occurs when using a trial version of a WinForms control in a WPF application.