Mapping interactive 2D UI onto 3D in WPF

There’s been a super exciting update in WPF’s 3D arena that’s been recently announced that I want to talk about here, but before I get to it, some background to set the stage.

I was heavily involved in the initial definition and design of the 3D aspects of WPF, and our overriding goal was to provide a continuity of programming model from 2D into 3D, both in the “media” aspects of it (having media, imaging, 2D graphics and 3D graphics types behave in a compatible, naturally interoperable and composable manner), and in the UI aspects of it, so that 3D would fit nicely into the overall WPF UI paradigm.

Overall what we produced in WPF v1 is, I believe, unique in the industry with respect to its integration as described above.  For instance, it’s straightforward to construct a full 2D interface in WPF, and then readily have it used as the material for a 3D object, including any live animations and video that the 2D UI might have.  This is a uniquely powerful and enabling concept and capability that WPF provides.

However, given people and time constraints in the development of WPF, we weren’t able to do all we had wanted to on some of the UI aspects of integration.  In particular, in the first version of WPF, we don't provide out-of-the-box support to interact with 2D UI that’s mapped onto 3D objects.  Thus, I couldn’t press a button that’s part of a 2D UI mapped onto a 3D object, and have the app automatically recognize the button as pressed (at least not without an awful lot of app-specific work).

Not surprisingly, since 2D and 3D integration is really encouraged by WPF, this missing piece has become one of the most desired features for the 3D feature set.  And thanks to the hard (and very clever) work of Kurt Berglund and others on the WPF 3D team, we now have an add-on that you can download and use in your apps to provide interactive 2D on 3D in a very nice way. 

So, without further ado, go read about Interacting with 2D on 3D in WPF.  And watch the accompanying Channel 9 video, download it, grok it, use it in your apps, etc.

I’ll resist the temptation to add some XAML here to show just how powerful and straightforward it is, since there’s a great sample and explanation already available at the link above.

Comments

• Anonymous
  January 11, 2007
  Greg, What are the limitations of the technique used to provide 2D interaction in a 3D scene?  I have viewed the available material which discusses how it works and how to use it, but am curious about how this feature could be improved.  If you were involved in making this functionality a part of WPF proper, what shortcomings would you address? Thanks for any insights.

• Anonymous
  January 15, 2007
  This site might be usefull as people start adding more to it, its a d3d 10 and wpf wiki. http://www.dxten.com