Mixed Media – Audio & Video – Windows Store vs. Windows Phone 8
As everyone knows, with the release of Windows 8 and Windows Phone 8 we have brought the two platforms closer together than ever before. The keyword here is “closer”. There are still some fairly major implementation and terminology differences between the two platforms particularly when working with digital audio and video. Hopefully this article will help you map and understand the media technologies between the two platforms. In this article I’m assuming that you are familiar with the media technologies on one platform or the other and just need some hints how to migrate your code to the other platform.
Audio / Video Playback Technologies
Audio / Video Playback – Technology Map |
|
Windows Store |
Windows Phone 8 |
|
MediaElement (XAML)
|
MediaElement (XAML) |
|
IMFMediaEngine (C++ only) |
IMFMediaEngine (C++ only) (Subset)
|
|
WASAPI (C++ only) |
WASAPI (C++ only) (Subset)
|
|
XAudio2 (C++ only) |
XAudio2 (C++ only)
|
The MediaElement will likely be your “go to” control on both platforms for playing audio and video. There are a few differences though. You can’t add audio or video effects (plug-ins) to the MediaElement on the phone. The DRM implementation is also quite different between the two platforms. If you are not doing anything “fancy” you should be able to port your MediaElement implementation with no problem.
The MediaEngine (different from the MediaElement discussed above) is new for both platforms in version 8. On both platforms the MediaEngine can only be used from C++ and is not currently supported from managed code. The phone also has slightly reduced functionality. On the phone the MediaEngine only supports frame-server mode. Also keep in mind that on the phone hardware acceleration will always be used.
WASAPI and XAudio2 are also supported on both platforms. Unfortunately at this time these audio only technologies can only be called from C++. They are not currently supported from managed code. While the full XAudio2 functionality is supported on both platforms keep in mind that XAudio2 only supports uncompressed PCM audio on the phone. It is extremely difficult to use XAudio2 with compressed media formats on the phone at this time. The phone also only supports a subset of the WASAPI Core Audio APIs as well.
References |
||||
Windows Store app |
Windows Phone 8 app |
|||
|
Native audio APIs for Windows Phone 8
|
||||
|
Supported Microsoft Media Foundation APIs for Windows Phone 8
|
||||
Background Audio
Background Audio – Technology Map |
|
Windows Store (XAML / HTML5) |
Windows Phone 8 (XAML) |
|
MediaElement / Audio tag
|
BackgroundAudioPlayer |
|
msAudioCategory="BackgroundCapableMedia" |
AudioPlayerAgent
|
|
Windows.Media.MediaControl |
Universal Volume Control (UVC)
|
|
Media Foundation Source (C++ only) |
MediaStreamSource
|
The big difference playing background audio in a Windows Store app vs. Windows Phone 8 app is the use of the BackgroundAudioPlayer on the Phone and the MediaElement / Audio tag for Windows Store apps. The BackgroundAudioPlayer is a static class that can be used without needing instantiation. The MediaElement and Audio tags must be defined in markup before they can be used.
For your Windows Store app you must set MsAudioCategory=”BackgroundCapableMedia”. For your phone app you need to create an AudioPlayerAgent that will run in the background when your app gets swapped out.
The final thing to note is that on both platforms you need to handle user input from the lock screen media controls. On the phone we call these controls the “Universal Volume Control” (UVC) and in your Store app we call them the “media controls”. Different terms but the same basic functionality.
References |
|
Windows Store app |
Windows Phone 8 app |
|
How to play audio in the background (Windows Store apps using JavaScript and HTML) |
Background audio overview for Windows Phone
|
Sharing Media
Sharing Media – Technology Map |
|
Windows Store (XAML / HTML5) |
Windows Phone 8 (XAML) |
|
DataTransferManager class |
ShareMediaTask Class
|
The big difference here is that with Windows Store apps the user initiates the sharing process using the “share charm”. On the phone your app will need to invoke the ShareMediaTask directly from within your code.
In both cases you will likely never get confirmation that your data was successfully shared. In essence it’s a “fire and forget” operation.
Keep in mind that on the phone your app will be swapped out and may get shut down when the app you are sharing with takes up the full screen. Because of this you may need to save state.
References |
|
Windows Store app |
Windows Phone 8 app |
|
Sharing and receiving content (Windows Store apps using JavaScript and HTML) (Windows) |
How to use the share media task for Windows Phone
|
Follow us on Twitter @wsdevsol.
Comments
Anonymous
March 19, 2013
No mention of MediaStreamSource, which still works pretty well on WP8. In the future, will MSS be phased out on WP8 or introduced in Win8 Store APIs?Anonymous
October 14, 2013
Does MediaElement on Windows Phone 8 support hardware acceleration？Anonymous
August 14, 2014
Hello, In your post you mention that "You can’t add audio or video effects (plug-ins) to the MediaElement on the phone". It looks like on WP 8.1 this is now possible, at least for video effects, as demonstrated in the Media Extensions sample. However, I have tried to follow the same approach as the Media Extensions sample to implement my own audio effect on WP1 without success. Video effects work fine, though. More details in my post at social.msdn.microsoft.com/.../audio-mft-and-mediaelement Do you happen to know why audio effects don't work on the phone, while video effects do? Thanks for your time, IanierAnonymous
September 05, 2014
Any good references to recording audio would be helpful on this page. Yes, I can search, but I'm wondering if there is something the author recommends.Anonymous
May 28, 2015
I'm trying to figure out how one would distribute an MFT for use with Wnidows Store/Phone/Universal apps. Can they be distributed through the Store or do people have to somehow install them out of band? Or worse, would they have to be bundled with every app that wants to use it? I've got a video codec that I wrote and I'd like existing Windows Store apps to be able to use it. Since it normally goes in avi, mp4/mov or mkv files, it makes no sense to have a special player for it.