Mixed Media – Audio & Video – Windows Store vs. Windows Phone 8

As everyone knows, with the release of Windows 8 and Windows Phone 8 we have brought the two platforms closer together than ever before. The keyword here is “closer”. There are still some fairly major implementation and terminology differences between the two platforms particularly when working with digital audio and video. Hopefully this article will help you map and understand the media technologies between the two platforms. In this article I’m assuming that you are familiar with the media technologies on one platform or the other and just need some hints how to migrate your code to the other platform.

 

Audio / Video Playback Technologies

 

Audio / Video Playback – Technology Map
Windows Store	Windows Phone 8
MediaElement (XAML)	MediaElement (XAML)
IMFMediaEngine (C++ only)	IMFMediaEngine (C++ only) (Subset)
WASAPI (C++ only)	WASAPI (C++ only) (Subset)
XAudio2 (C++ only)	XAudio2 (C++ only)

 

The MediaElement will likely be your “go to” control on both platforms for playing audio and video. There are a few differences though. You can’t add audio or video effects (plug-ins) to the MediaElement on the phone. The DRM implementation is also quite different between the two platforms. If you are not doing anything “fancy” you should be able to port your MediaElement implementation with no problem.

The MediaEngine (different from the MediaElement discussed above) is new for both platforms in version 8. On both platforms the MediaEngine can only be used from C++ and is not currently supported from managed code. The phone also has slightly reduced functionality. On the phone the MediaEngine only supports frame-server mode. Also keep in mind that on the phone hardware acceleration will always be used.

 

WASAPI and XAudio2 are also supported on both platforms. Unfortunately at this time these audio only technologies can only be called from C++. They are not currently supported from managed code. While the full XAudio2 functionality is supported on both platforms keep in mind that XAudio2 only supports uncompressed PCM audio on the phone. It is extremely difficult to use XAudio2 with compressed media formats on the phone at this time. The phone also only supports a subset of the WASAPI Core Audio APIs as well.

 

References
Windows Store app		Windows Phone 8 app
Win32 and COM for Windows Store apps (multimedia)		Native audio APIs for Windows Phone 8
IMFMediaEngine interface (Windows)		Supported Microsoft Media Foundation APIs for Windows Phone 8
MediaElement class (Windows)		MediaElement Class

 

Background Audio

 

Background Audio – Technology Map
Windows Store (XAML / HTML5)	Windows Phone 8 (XAML)
MediaElement / Audio tag	BackgroundAudioPlayer
msAudioCategory="BackgroundCapableMedia"	AudioPlayerAgent
Windows.Media.MediaControl	Universal Volume Control (UVC)
Media Foundation Source (C++ only)	MediaStreamSource

 

The big difference playing background audio in a Windows Store app vs. Windows Phone 8 app is the use of the BackgroundAudioPlayer on the Phone and the MediaElement / Audio tag for Windows Store apps. The BackgroundAudioPlayer is a static class that can be used without needing instantiation. The MediaElement and Audio tags must be defined in markup before they can be used.

 

For your Windows Store app you must set MsAudioCategory=”BackgroundCapableMedia”. For your phone app you need to create an AudioPlayerAgent that will run in the background when your app gets swapped out.  

 

The final thing to note is that on both platforms you need to handle user input from the lock screen media controls. On the phone we call these controls the “Universal Volume Control” (UVC) and in your Store app we call them the “media controls”. Different terms but the same basic functionality.

 
 

References
Windows Store app	Windows Phone 8 app
How to play audio in the background (Windows Store apps using JavaScript and HTML)	Background audio overview for Windows Phone

 

Sharing Media

 

Sharing Media – Technology Map
Windows Store (XAML / HTML5)	Windows Phone 8 (XAML)
DataTransferManager class	ShareMediaTask Class

 

The big difference here is that with Windows Store apps the user initiates the sharing process using the “share charm”. On the phone your app will need to invoke the ShareMediaTask directly from within your code.

 

In both cases you will likely never get confirmation that your data was successfully shared. In essence it’s a “fire and forget” operation.

 

Keep in mind that on the phone your app will be swapped out and may get shut down when the app you are sharing with takes up the full screen. Because of this you may need to save state.

 

References
Windows Store app	Windows Phone 8 app
Sharing and receiving content (Windows Store apps using JavaScript and HTML) (Windows)	How to use the share media task for Windows Phone

 

Follow us on Twitter @wsdevsol.

Comments

• Anonymous
  March 19, 2013
  No mention of MediaStreamSource, which still works pretty well on WP8. In the future, will MSS be phased out on WP8 or introduced in Win8 Store APIs?

• Anonymous
  October 14, 2013
  Does MediaElement on Windows Phone 8 support hardware acceleration？

• Anonymous
  August 14, 2014
  Hello, In your post you mention that "You can’t add audio or video effects (plug-ins) to the MediaElement on the phone". It looks like on WP 8.1 this is now possible, at least for video effects, as demonstrated in the Media Extensions sample. However, I have tried to follow the same approach as the Media Extensions sample to implement my own audio effect on WP1 without success. Video effects work fine, though. More details in my post at social.msdn.microsoft.com/.../audio-mft-and-mediaelement Do you happen to know why audio effects don't work on the phone, while video effects do? Thanks for your time, Ianier

• Anonymous
  September 05, 2014
  Any good references to recording audio would be helpful on this page. Yes, I can search, but I'm wondering if there is something the author recommends.

• Anonymous
  May 28, 2015
  I'm trying to figure out how one would distribute an MFT for use with Wnidows Store/Phone/Universal apps.  Can they be distributed through the Store or do people have to somehow install them out of band?  Or worse, would they have to be bundled with every app that wants to use it?  I've got a video codec that I wrote and I'd like existing Windows Store apps to be able to use it. Since it normally goes in avi, mp4/mov or mkv files, it makes no sense to have a special player for it.