MediaPlayer and MediaStreamSource on Hololens 2

Question

Hi there,

I have a working MediaPlayer based decoder that consumes livestreamed H264 video from incoming frames, using MediaStreamSource and responding to requests for samples from MediaStreamSource.SampleRequested. This is integrated into Unity in order to display these frames on a IDirect3DSurface.

I am happily doing 30fps on my desktop, but when running on a Hololens 2 I am only visually seeing about maybe 5fps.

That being said, I have numerous metrics inside the MediaPlayer code that show the sample requests, sample resolutions and VideoFrameAvailable callbacks are all still happening at 30fps. I am not losing frames, it is more like only every X frame actually gets displayed, and intermediate ones are not. Finally, if I use a test MediaSource created from a URI and run through the same pipeline, I do see 30fps on the Hololens 2, so that rules out issues with my surface / display logic.

I do see a fairly large number of _com_errors raised inside d3d11.dll CDecodeContext::BeginFrame when the debugger is attached, but these never make it up to my code and perhaps are harmless? ref: https://social.microsoft.com/Forums/azure/fr-FR/f76a80db-3bf0-49b1-8c4f-4d3b90c03f94/how-to-track-down-comerrors?forum=winappswithnativecode

Are there any known issues with this kind of streaming that are specific to the Hololens 2?

Answer 1

Hololens is currently not supported in the Q&A forums, the supported products are listed over here https://learn.microsoft.com/en-us/answers/products (more to be added later on).

You can ask the experts in the dedicated Hololens forums over here:
https://stackoverflow.com/questions/tagged/hololens

https://stackoverflow.com/search?q=windows-mixed-reality

(Please don't forget to accept helpful replies as answer)

Best regards,
Leon