Share via

HTML5 Video Captioning

  • Article

The promise of HTML5 is a Web that works for everyone. Media accessibility through captioning is an important part of that promise and an area that is still emerging through standards work. Using the proposed HTML5 track element, developers can add captioning to HTML5 video by pointing to a caption file that contains a written representation of the dialog or actions in the video. Once the standards for captioning converge, there will be less need for external add-ons to publish accessible video content.

Still image from a video showing captions

The following HTML fragment shows how the track element works:

<video>

<source type="video/mp4" src="video_file" >

<track src="captions_file" label="English captions" kind="captions" srclang="en-us" default >

<track src="descriptions_file" label="English description" kind="descriptions" srclang="en-us" >

</video>

The W3C HTML5 specification allows for multiple caption formats. Internet Explorer 10 in the Windows Developer Preview supports the <track> element, but does not download or display any caption files yet. We use this implementation to prototype how different caption formats can be supported.

Last year at the W3C TPAC meeting, media accessibility was discussed with a proposal for the accessibility requirements that HTML5 should support. Recently, a Web Media Text Tracks Community Group was created to further work on captioning solutions for the Web.

Even though we are not yet at the stage of a final HTML5 spec, it is stabilizing. In order to provide better feedback on <track> and caption file formats, the Internet Explorer and TwC Accessibility teams partnered to build a prototype that uses the element to display captions and descriptions on a video.

We wanted to test a scenario were multiple formats are used to gather more feedback, so we built the demo in such a way that both TTML-1.0 and WebVTT can be used. WebVTT originated from W3C discussions last year after a need for simple caption authoring was identified. TTML is already an established standard for full-featured video captioning and supported in Adobe Flash and Microsoft Silverlight. It is used on Netflix, Hulu and other sites that display broadcast content.

Screen shot of the prototype that displays TTML and WebVTT video captions and descriptions

The prototype uses the video and track elements to show how TTML and WebVTT formats can be used to enable captions and descriptions on a video playback. The prototype supports all HTML5-enabled browsers.

You can interact with the prototype at the HTML5 Labs site. Use View Source or your browser’s developer tools to inspect the Javascript that parses the TTML and WebVTT formats.

As always, we welcome your feedback.

—Frank Olivier, Senior Program Manager, Internet Explorer

Comments

  • Anonymous
    October 12, 2011
    Please - made foolscreen for HTML5 video

  • Anonymous
    October 12, 2011
    This is great! Only Firefox supports the track element but also no caption file downloaiding and displaying. Will IE support webvtt or ttml? pleaase do webvtt because every other browser will be.  

  • Anonymous
    October 12, 2011
    The comment has been removed

  • Anonymous
    October 12, 2011
    The comment has been removed

  • Anonymous
    October 12, 2011
    Nice work on the TextTrack implementation and WebVTT parser (www.interoperabilitybridges.com/.../jquery.msft.video-access.js). Will this be open sourced as a library or just added into IE10? I'd love to use something like it in http://mediaelementjs.com/

  • Anonymous
    October 12, 2011
    Full screen, full screen, full screen!

  • Anonymous
    October 13, 2011
    Could you change the style slightly to avoid using only mozilla- or webkit- prefixed CSS attributes? It looks ugly in non webkit or mozilla based browsers.

  • Anonymous
    October 13, 2011
    Will there be fallback compatibility support for IE6 ??

  • Anonymous
    October 13, 2011
    Hello Internet Explorer Team, I have a question that needs answering fast please answer.. Internet Explorer 9 won't show the images on the bottom of the web page at play.clubpenguin.com/index_small_screen.php  I cleared my cache and cookies about 2 times.. on my other computer that is a desktop the images shows up but not on my laptop computer for some reason.. :( please help

  • Anonymous
    October 13, 2011
    Do i need to clear ActiveX filtering and tracking protection cache also?

  • Anonymous
    October 13, 2011
    The comment has been removed

  • Anonymous
    October 13, 2011
    Since commenting is closed on old posts, like blogs.msdn.com/.../testing-multiple-versions-of-ie-on-one-pc.aspx , I want to ask Microsoft to stop being evil, and allow http://www.spoon.net/ to host Internet Explorer versions, for testing purposes. Forcing me to install 3 virtual machines to test IE8, IE7 and IE6 on my Windows 7 laptop with IE9 is evil. The 3 VMs use 29 GB of disk space, and I only have a 80 GB SSD. I wouldn't have commented here, if the VMs would have worked properly. But the latest Vista virtual machine crashes with blue screen as soon as I visit two or three pages with IE 7. I uploaded the crash dumps to www.mediafire.com ; the small file contains minidumps and associated files, and two blue screen screenshots; the two large files are two full memory dumps, if anyone is interested. I downloaded the VMs yesterday from www.microsoft.com/.../details.aspx (Windows Virtual PC VHDs for testing websites with different Internet Explorer versions [Last Updated: 8/16/11]). To conclude, please allow Spoon.net to host Internet Explorer.

  • Anonymous
    October 14, 2011
    Although I appreciate the IE team making strides on video HTML5 implementations, the underlying problem is a lack of a uniformity in the file formats among browsers. Please add support for Ogg Theora and WebM

  • Anonymous
    October 14, 2011
    @Nelson - You can control how the blog home page is displayed by clicking on the Excerpt icon next to the Sort By options. That'll reduce your page load. As to comment submission, the dev team is actively working on fixes for the comment submission at this time. We hope to have a patch out soon.

  • Anonymous
    October 14, 2011
    On this W3C test w3c-test.org/.../paused_true_during_pause.html IE passes the test but continue the playback while it should remain at the paused state. Other browsers don't show this behavior. OAN, IE fails all the tests laid by Apple w3c-test.org/.../Apple

  • Anonymous
    October 20, 2011
    The comment has been removed