How to await a MediaElement (PlaySound in Windows 8)

[This post is part of a series How to await a storyboard, and other things]

 

Let’s look at making MediaElement awaitable. This is the kind of idiom I’d like to use:
  

Try

    Await mediaElement1.OpenAsync(New Uri("ms-appx:///Assets/boooo.mp3"))

    Await mediaElement1.PlayAsync()

Catch ex As Exception

    ...

End Try
  

Q. Why do I want to await it? Why not just set its AutoPlay property to True, then set its Source property, and let it go?
  

A. Well, AutoPlay is fine for simple scenarios. But it doesn’t compose in the nice way that await does. It doesn’t let me use my familiar exception handling blocks - inside I’d have to sign up to a MediaFailed event handler. And it doesn’t let me compose in other interesting ways, e.g.
  

Await Task.WhenAll(storyboard1.PlayAsync(), mediaElement1.Play())
  

 

ALERT. MediaElement must be in the visual tree. Typically that means you create it as a child of your current Xaml page. If you want it to play globally, i.e. independent of which page is being shown, read this thread: https://social.msdn.microsoft.com/Forums/en-US/winappswithcsharp/thread/241ba3b4-3e2a-4f9b-a704-87c7b1be7988/

 

Jumping straight to the answer, here’s my attempt at code that supports awaitability. There may be bugs in it: it’s complex, and this is my first attempt. Requirements: neither of these operations is re-entrant: i.e. if other operations are performed on a given media element while either one is in progress, it might go awry.
  

<Extension> Function OpenAsync(media As MediaElement, uri As Uri,
                               Optional cancel As CancellationToken = Nothing) As Task

    Dim tcs As New TaskCompletionSource(Of Object)

 

    If cancel.IsCancellationRequested Then

        tcs.SetCanceled()

        Return tcs.Task

    End If

 

    If media.CurrentState = MediaElementState.Buffering OrElse

                     media.CurrentState = MediaElementState.Opening OrElse

                     media.CurrentState = MediaElementState.Playing Then

        tcs.SetException(New Exception("MediaElement not ready to open"))

        Return tcs.Task

    End If

 

    Dim lambdaOpened As RoutedEventHandler = Nothing

    Dim lambdaChanged As RoutedEventHandler = Nothing

    Dim lambdaFailed As ExceptionRoutedEventHandler = Nothing

    Dim cancelReg As CancellationTokenRegistration? = Nothing

    Dim removeLambdas As Action = Sub()

                                      RemoveHandler media.MediaOpened, lambdaOpened

                                      RemoveHandler media.MediaFailed, lambdaFailed

                                      RemoveHandler media.CurrentStateChanged, lambdaChanged

                                      If cancelReg.HasValue Then cancelReg.Value.Dispose()

                                      removeLambdas = Sub() Return

                                      ' in case two lambdas get fired one after the other

                                  End Sub

 

    lambdaOpened = Sub()

                       removeLambdas()

                       tcs.TrySetResult(Nothing)

                   End Sub

    lambdaFailed = Sub(s, e)

                       removeLambdas()

                       tcs.TrySetException(New Exception(e.ErrorMessage))

                   End Sub

    lambdaChanged = Sub()

                        If media.CurrentState <> MediaElementState.Closed Then Return

                        removeLambdas()

                        tcs.TrySetCanceled()

                    End Sub

 

    AddHandler media.MediaOpened, lambdaOpened

    AddHandler media.MediaFailed, lambdaFailed

    AddHandler media.CurrentStateChanged, lambdaChanged

 

    media.Source = uri

 

    If Not tcs.Task.IsCompleted Then

        ' The above condition guards against lambas being invoked by Source assignment

        cancelReg = cancel.Register(

            Sub()

                Dim dummy = media.Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal,

                          Sub()

                              media.ClearValue(MediaElement.SourceProperty)

                          End Sub)

            End Sub)

    End If

 

    Return tcs.Task

End Function

 

 

 

 

<Extension> Function PlayAsync(media As MediaElement,
                               Optional cancel As CancellationToken = Nothing) As Task

    Dim tcs As New TaskCompletionSource(Of Object)

 

    If cancel.IsCancellationRequested Then

        tcs.SetCanceled()

        Return tcs.Task

    End If

 

    If media.CurrentState <> MediaElementState.Paused Then

        tcs.SetException(New Exception("MediaElement not ready to play"))

        Return tcs.Task

    End If

 

    Dim lambdaEnded As RoutedEventHandler = Nothing

    Dim lambdaChanged As RoutedEventHandler = Nothing

    Dim cancelReg As CancellationTokenRegistration? = Nothing

    Dim removeLambdas As Action = Sub()

                                      RemoveHandler media.MediaEnded, lambdaEnded

                                      RemoveHandler media.CurrentStateChanged, lambdaChanged

                                      If cancelReg.HasValue Then cancelReg.Value.Dispose()

                                      removeLambdas = Sub() Return

                                  End Sub

 

    lambdaEnded = Sub()

                      removeLambdas()

                      tcs.TrySetResult(Nothing)

                  End Sub

    lambdaChanged = Sub()

                        If media.CurrentState <> MediaElementState.Stopped Then Return

                        removeLambdas()

                        tcs.TrySetCanceled()

                    End Sub

 

    AddHandler media.MediaEnded, lambdaEnded

    AddHandler media.CurrentStateChanged, lambdaChanged

 

    media.Play()

 

    If Not tcs.Task.IsCompleted Then

        cancelReg = cancel.Register(

            Sub()

                Dim dummy = media.Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal,

                          Sub()

                              media.Stop()

                          End Sub)

            End Sub)

 

    End If

 

    Return tcs.Task

End Function

 

What is the interactive behavior of MediaElement?

There’s no way I could make MediaElement awaitable without knowing the details of how it behaves. I did my PhD on the “pi calculus”, a sort of mathematical algebra for describing and specifying interactive behaviors. Now I’m not going to delve into mathematics here, but I believe that software engineering requires the same careful approach when you get down to the edge cases of async. Turning a complex event-based thing into a Task-based thing is one of those edge cases.

 

When studying the interactive behavior of a thing, then you HAVE to understand its complete state diagram. You have to understand in which states you’re allowed to trigger which stimuli (e.g. invoking the Play() method), and what their effect will be. You have to understand in which states the thing might have its own internal stimuli (e.g. reaching the end of a piece of media) and what effects these will have. If the documentation fails to specify all this, then you'll have to discover it by yourself.

I’m not a XAML expert. I don’t know the full details of how MediaElement behaves. But from experiment, this is what I believe is its state diagram:

 

State	Possible internal transitions	Upon setting Source	Upon reset of Source	Upon invoking Play()	Upon invoking Stop()
Closed		-> Opening
Opening	-> Paused (MediaOpened) -> Closed (MediaFailed)	?	-> Closed
Paused		-> Opening	-> Closed	-> Buffering -> Playing	-> Stopped
Buffering	-> Playing	?	?		-> Stopped
Playing	-> Buffering -> Paused (MediaEnded)	?	?		-> Stopped
Stopped		-> Opening	?	-> Buffering -> Playing

 

Read the table like this: “If the media element is currently in <State>, then it might transition “->” to another state either due to its internal workings or due to some method being invoked. If it does transition, then first its CurrentState property is changed, then it may fire a specified event on the UI thread, then it fires the CurrentStateChanged event on the UI thread.”
  

The media element starts in state “Closed” upon construction. If its Source property is set in XAML, then that has the same effect as setting the source property in code. The column for “reset of source” refers to invoking the method “mediaElement1.ClearValue(MediaElement.SourceProperty)”. That is the only way I have found to completely reset the Source property from code.
  

I noticed that the transition “Opening -> Closed (MediaFailed)” can take up to 10 seconds in the case of an https:// uri which either doesn’t exist (returns an error http code) or where your app lacks Internet Client permissions.
  

I haven’t tried to characterize every possible method than can be invoked on mediaElement. I only characterized the ones needed to support my goals.
  

Conclusions

Does this code look way more complicated to you than it should? It does to me. To be clear, the use of the code is very simple
  

Await mediaElement1.OpenAsync(New Uri("ms-appx:///Assets/boooo.mp3"))

Await mediaElement1.PlayAsync()

 

It’s just the implementation of awaitability-support that’s complex. Why so complex? Actually, my initial attempt was much simpler. It merely played a media element which already had its Source property set to the right value, and had already opened it successfully.
  

<Extension> Function PlayAsync1(media As MediaElement) As Task

    Dim tcs As New TaskCompletionSource(Of Object)

    Dim lambda As RoutedEventHandler = Sub()

                                           RemoveHandler media.MediaEnded, lambda

                                           tcs.SetResult(Nothing)

                                       End Sub

    AddHandler media.MediaEnded, lambda

    media.Play()

    Return tcs.Task

End Function

 

It got more complex because (0) I wanted to support setting the source property, (1) I wanted to support cancellation, (2) I wanted to figure out the complete state diagram, to know in which states it’s valid to call PlayAsync, and (3) I wanted to know whether PlayAsync might ever end in failure. People often complain that “the Play() method doesn’t do anything”, and the answer is that Play() is only valid after MediaOpened has fired.
  

My analysis isn’t complete. I don’t know what happens if you set the source at the wrong time. What would happen if you did “Dim t1 = media1.OpenAsync(uri1), t2 = media1.OpenAsync(uri2)” ? Would it end up with a random one of the two URIs opened? Or would it end up in some broken internal state? Or what happens if, in response to a button-click, you did “Await media1.OpenAsync(uri)”, and the user impatiently clicks the button a second time while it was still opening the first one? Would it finish the first open operation? or abort it and start the second open operation? Or would it end in some broken state?
  

  

The ultimate problem is that MediaElement has two very different uses – either as a full-blown media playing control with start/stop/seek controls, or just for doing PlaySoundAsync(). The first use isn’t much like a task at all. That’s because there’s no single obvious completion event, and because it isn’t monotonic. A monotonic state diagram is one where you never return to an earlier state. Readonly variables are also monotonic, since they transition from “unassigned” to “assigned-with-value”. Monotonic things are much easier to program against. Compare the state-diagram for MediaElement above, with the simpler state diagram for Tasks that come out of async methods. It’s easy to know where you are with tasks.
  

State	Possible internal transitions
WaitingForActivation	-> RanToCompletion -> Canceled -> Faulted
RanToCompletion
Canceled
Faulted

 

I do wonder whether it might be best to abandon awaitability of MediaElement itself, and encapsulate it into a simpler API like PlaySoundAsync. But this then would have its own problems: how do I pause it? how do I declare it in XAML? how do I open it just once but then play it multiple times? What forms of uri does it support? How to set the volume? How does it find the visual-tree? My encapsulation might end up just as complicated as MediaElement!
  

Function PlaySoundAsync(uri As Uri, Optional cancel As CancellationToken) As Task

    ...

End Function

Comments

• Anonymous
  August 24, 2016
  I just want to make my mediaelement:file = Await folder.GetFileAsync(f)stream = Await file.OpenAsync(FileAccessMode.Read)media.SetSource(stream, file.ContentType)cancelable. a simple solution?