Unlocking the power of HTML5

Sounds form the background of our life. Today the HTML5 <audio> element enables Web developers to embed sounds in their applications. The flexibility of the control coupled with the integration with the rest of the platform allows several scenarios, from simple sound effects to background audio to gaming experiences to more sophisticated audio engines.

This blog post walks through some of the best practices for using the <audio> tag in your Web applications, and includes useful tips from real-world sites.

Adding an audio element to your page

The very first step is to add the audio element to your page. You can do this by declaring an <audio> tag in your markup, by instantiating a new audio element in the JavaScript code, or by embedding the audio stream in the page:

<audio src="audio/sample.mp3" autoplay>

</audio>

Run live demo

var audio = document.createElement("audio");

if (audio != null && audio.canPlayType && audio.canPlayType("audio/mpeg"))

{

audio.src = "audio/sample.mp3";

audio.play();

}

Run live demo

<audio src="data:audio/mpeg,ID3%02%00%00%00%00%..." autoplay>

</audio>

Run live demo

The first approach allows you to initialize the audio components during the page load. The second approach gives you more flexibility and better management of the network flow, as it defers the loading of the audio clip to a specific time during the application lifecycle. The third approach (less recommended) consists in embedding the audio files as data-uri in the page, reducing the number of requests to the server.

Note that you can play an audio element generated by JavaScript even if it’s not been actually added to the DOM tree (like in the code snippet above). However adding the audio element to the page will allow you to display the default control bar.

Although not covered in this post, you can support more than one audio file format. Also, if you are hosting the audio files on your server, remember to register the MIME type for mp3 files (“audio/mpeg”) on the server side. Here, for instance, is the setting on Internet Information Services (IIS).

Preloading the audio before playing

Once you have your audio element, you can choose the best preloading strategy. The HTML5 <audio> specification describes a preload property with three possible values:

• “none”: hints to the user agent that either the author does not expect the user to need the media resource, or that the server wants to minimize unnecessary traffic.
  If your scenario is a podcast blog with an audio file for each post, this option works particularly well, as it reduce the initial preload bandwidth. Once the user plays the file (either through the default visual controls or the JavaScript methods load() or play() ), the browser will start fetching the audio stream.
• “metadata”: hints to the user agent that the author does not expect the user to need the media resource, but that fetching the resource metadata (dimensions, duration, etc.) is reasonable.
  This option is recommended if you are building an audio player control and you need basic information about the audio clip, but don’t need to play it yet.
• “auto”: hints to the user agent that the user agent can put the user's needs first without risk to the server, up to and including optimistically downloading the entire resource.
  If you are building a game, this approach is probably the best one, as it allows you to preload all the audio clips before actually starting the game experience.

Note that when you set the src property of the audio element programmatically, the browser will set the preload property – unless otherwise specified – to “auto.” For this reason, if your scenario needs a different value, make sure to specify it in a line of code before setting the src.

You can preview the impact over the network of these three options by running this page using the F12 Developer Tools (Network Tab). For debug purposes, you can simulate new calls and disable the local cache by checking the “Always refresh from sever” menu.

preload=none:

preload=metadata:

preload=auto:

While this property is great for the initialization phase, you might also need to know when the browser actually downloaded the audio clip and is ready to play it. You can get this information by listening to the “canplaythrough” event; this event is called by the User Agent once it estimates that if playback were to be started now, the media resource could be rendered at the current playback rate all the way to its end without having to stop for further buffering.

var audio = document.createElement("audio");

audio.src = "audio/sample.mp3";

audio.addEventListener("canplaythrough", function () {

alert('The file is loaded and ready to play!');

}, false);

Run live demo

Loops

Another frequent request for scenarios with audio is the ability to loop a sound clip. With the HTML5 <audio>, you can do this using the “loop” property; this setting will loop your clip forever, or until the user or the application activates the pause() audio control.

<audio src="audio/sample.mp3" autoplay loop>

</audio>

Run live demo

Another approach to loop an audio file is to programmatically call the play() method when the audio clip ends; doing so will allow you eventually to manage the delay between one loop and the other.

var audio = document.createElement("audio");

audio.src = "piano/3C.mp3";

audio.addEventListener('ended', function () {

// Wait 500 milliseconds before next loop

setTimeout(function () { audio.play(); }, 500);

}, false);

audio.play();

Run live demo

Note that any play() call executed on the audio element before the sound actually ended won’t have any effect. If you are interested to “cancel and restart” the current sound, you will need to reset the currentTime.

var audio = null;

audio = document.createElement("audio");

audio.src = "piano/3C.mp3";

audio.addEventListener('ended', function () {

audio.play();

}, false);

function play() {

audio.play();

}

function restart() {

audio.currentTime = 0;

audio.play();

}

Run live demo

Multiple audio tags

If your scenario needs the same audio file to be played several times concurrently (that is, with overlapping sounds), you can achieve this result by creating multiple audio tags pointing to the same file. Obviously the same approach also works if you are using different audio files at the same time. As we explained earlier in this post, you can either add those programmatically or by instantiating them in the markup.

The following code snippet shows how to load and play multiple audio files using markup. The audio samples all have the same length; at the end of the execution, they will loop starting from the beginning. As you play them in Internet Explorer 9, you can notice that they are automatically synchronized throughout various loops. You will notice that the combination of these 5 sounds will play like the audio file used in the previous demo (“sample.mp3”).

<body>

<audio src="audio/Bass.mp3" autoplay loop>

</audio>

<audio src="audio/Drum.mp3" autoplay loop>

</audio>

<audio src="audio/Crunch.mp3" autoplay loop>

</audio>

<audio src="audio/Guitar.mp3" autoplay loop>

</audio>

<audio src="audio/Pizzicato.mp3" autoplay loop>

</audio>

</body>

Run live demo

While this approach is very simple and straightforward, in most scenarios developers prefer to create the audio clips programmatically. The following code snippet shows how to add 3 audio clips dynamically using code. As you play them together, you will get the C Major chord!

AddNote("3C");

AddNote("3E");

AddNote("3G");

function AddNote(name) {

var audio = document.createElement("audio");

audio.src = "piano/" + name + ".mp3";

audio.autoplay = true;

}

Run live demo

This code pattern works on any browser and will allow you to build very compelling scenarios!

It’s important to keep in mind that as your application or game becomes more complex you might eventually reach two limits: the number of audio elements you can preload on the same page and the number of audio elements you can play at the same time.

These numbers depend on the browser and the capabilities of your PC. Based on my experience, Internet Explorer 9 can handle dozens of concurrent audio elements simultaneously with no issues. Other browsers don’t do as well – you might encounter evident delays and distortions as you play multiple files in a loop.

Synchronization strategies

Depending on the characteristics of the network, you should always consider the delay involved between adding the tag, getting the content and being ready to play it. In particular, when you are handling multiple files, each file might be ready to play earlier or later. Here, for instance, is a capture with the 3 files used previously, loaded from the local host.

As you can see in the Timings column, different files might be ready at a different time.

A very common synchronization strategy is to preload all the files first. Once they are all ready, you can quickly iterate through a loop and start playing them.

var audios = [];

var loading = 0;

AddNote("2C");

AddNote("2E");

AddNote("2G");

AddNote("3C");

AddNote("3E");

AddNote("3G");

AddNote("4C");

function AddNote(name) {

loading++;

var audio = document.createElement("audio");

audio.loop = true;

audio.addEventListener("canplaythrough", function () {

loading--;

if (loading == 0) // All files are preloaded

StartPlayingAll();

}, false);

audio.src = "piano/" + name + ".mp3";

audios.push(audio);

}

function StartPlayingAll() {

for (var i = 0; i < audios.length; i++)

audios[i].play();

}

Run live demo

Let’s bring everything together now! The following demo simulates a piano playing Frère Jacques (also known as Brother John, Brother Peter…or Fra Martino). The page starts fetching all the notes, showing the progress as they get preloaded on the client. Once they are all ready, the song starts and keeps playing in a loop.

Run live demo

Audio in real world sites

Now that we’ve seen the common patterns to handle multiple audio files, I’d like to highlight a few Web sites as examples of best practice uses of the <audio> tag.

Pirates Love Daises: www.pirateslovedaisies.com

In another blog post I talked about Pirates Love Daises, an awesome HTML5 game built by Grant Skinner. In addition to great game play and compelling visual effects, Grant’s team also developed a sophisticated audio library that plays several audio samples throughout the game. The main logic is encapsulated within the AudioManager class. As suggested earlier, before actually starting the game the site preloads all the audio clips and display the cumulative progress in the initial loading screen. The site takes also into consideration the case of network timeouts or errors occurred while downloading an audio file.

addAudioChannel:function(b,a,f){

var h=document.createElement("audio");

if(f!=true){

this.currAsset=h;

this.timeoutId=setTimeout($.proxy(this,"handleAudioTimeout"),e.AUDIO_TIMEOUT);

h.addEventListener("canplaythrough",$.proxy(this,"handleAudioComplete"),false);

h.addEventListener("error",$.proxy(this,"handleAudioError"),false)

}

h.setAttribute("id",a);

h.setAttribute("preload","auto");

$("<source>").attr("src",b).appendTo(h);

$("<source>").attr("src",b.split(".mp3")[0]+".ogg").appendTo(h);

document.body.appendChild(h)

}

,handleAudioComplete:function(b){

if(LoadedAssets.getAsset(b.target.id)!=true){

LoadedAssets.addAsset(b.target.id,true);

clearTimeout(this.timeoutId);

this.calculatePercentLoaded(true)

}

}

,handleAudioError:function(b){

trace("Error Loading Audio:",b.target.id);

LoadedAssets.addAsset(b.target.id,true);

clearTimeout(this.timeoutId);

this.calculatePercentLoaded(true)

}

,handleAudioTimeout:function(){

trace("Audio Timed Out:",this.currAsset.id);

LoadedAssets.addAsset(this.currAsset.id,true);

this.calculatePercentLoaded(true)

}

Run live demo

Grant is currently working on a Sound Library project that will allow developers to use their sound engine’s logic with any other application. Looking forward to that!

Firework (by Mike Tompkins): www.beautyoftheweb.com/firework

The Firework demo is particularly interesting, as it allow you to interact with several audio tracks at the same time, dynamically changing the volume of each track. Moreover, as you interact with the audio channels, the interface dynamically reacts to different inputs or settings.

This time the audio tags have been declared in the HTML markup (there are just 6 tracks). The progress is tracked programmatically by listening to the canplaythrough event. Once all audio files are ready to play, a loop goes through the list and starts playing.

video.addEventListener('canplaythrough', onCanPlayAudio, false);

for (var i = 0; i < 5; i++) {

var aud = document.getElementById("aud" + i);

targetVolumes.push(0);

aud.volume = 0;

audioTags.push({

"tag": aud,

"ready": false

});

aud.addEventListener('canplaythrough', onCanPlayAudio, false);

}

// Set audio/video tracks

document.getElementById("tompkins").src = MediaHelper.GetVideoUrl("Firework_3");

for (var i = 0; i < audioTracks.length; i++) {

document.getElementById("aud" + i).src = MediaHelper.GetAudioUrl(audioTracks[i]);

}

Run live demo

The developers in this case also made the decision to start with the volume set to 0 and to increase it dynamically to 1 as soon as the experience is ready to play. Depending on the quality of your audio card and drivers, this little trick reduces the likelihood of hearing an initial “knock” noise when the audio starts.

BeatKeep: www.beatkeep.net

The last scenario is probably the most complicated of the examples shown here. In this example, you can build your own songs using a beat machine and playing several audio clips in a loop. In this application, it’s critical to have perfect synchronization of the audio channels and an agile buffering system to load multiple clips.

The beat machine gives you full control over the tempo and the time signature; using sophisticated timer logic and binding model—the end result is a very smooth experience!

Conclusions

I encourage you to try all the samples and applications in this post using Internet Explorer 9 or other browsers and let us know what your experience was like! You can download all the sample code used in this article here.

If you want to learn more about the audio and video controls, I recommend that you watch the half-hour session from MIX “5 Things You Need To Know To Start Using <audio> and <video> Today” or read these interesting articles on the MSDN.

Thanks to DoubleDominant for the audio clips used in this blog post and to Grant Skinner and Archetype for their great HTML5 experiences.

—Giorgio Sardo | Sr. Technical Evangelist – HTML5 and Internet Explorer

Comments

• Anonymous
  May 13, 2011
  So, why doesn't this work?   <audio src="sonate.mid" autoplay> </audio>

• Anonymous
  May 13, 2011
  > addAudioChannel:function(b,a,f){,a,f){ Something is wrong there.

• Anonymous
  May 13, 2011
  @alvatrus: What is the format of the sonate.mid file and what MIME type does the server send when serving it? @Björn: Thanks. Typo corrected.

• Anonymous
  May 13, 2011
  The mp3 format is great though where is support for the Ogg Vorbis codec? As alvatrus points out why doesn't the audio element support the midi format? Interoperability automatically and inherently includes codecs in conjunction with the audio and video elements and IE9's implementation is NOT interoperable with other browsers nor is Safari.

• Anonymous
  May 13, 2011
  As part of the example, you use "audio/mp3" as the MIME type - "if (audio != null && audio.canPlayType && audio.canPlayType("audio/mp3"))" But later you state "audio/mpeg" as the MIME type - "remember to register the MIME type for mp3 files (“audio/mpeg”)" Be consistent.

• Anonymous
  May 13, 2011
  autoplay is not supported by mobile Safari currently! The playback needs to be triggered by the build in controls of the audio tagor a javascript function triggering the play() method after a user interaction.

• Anonymous
  May 13, 2011
  @JAB Creations: The ogg format is great though where is support for the MP3 codec? Interoperability automatically and inherently includes codecs in conjunction with the audio and video elements and Firefox's implementation is NOT interoperable with other browsers nor is Chrome. See? I can do it too!

• Anonymous
  May 13, 2011
  @PhistucK: I have updated the canPlayType call to use "audio/mpeg". I note, however, that browsers that can play MP3 files answer "maybe" to both canPlayType("audio/mpeg") and canPlayType("audio/mp3"). Browsers that cannot return "" to both calls. A test page for a collection of MIME types is at www.baccano.com/.../support.htm

• Anonymous
  May 13, 2011
  @Aethec You should consider directing your concerns to relevant blogs.

• Anonymous
  May 13, 2011
  Why does IE continue to support royalty, patented and DRM encumbered formats such as MP3? OGG Vorbis fixes these issues as it does not have any DRM, is royalty free, and is a completely open standard. By the way, on my own websites I will only ever use Vorbis and other open standards for audio, never so-called "standards" that have these issues.

• Anonymous
  May 13, 2011
  @Andrew If Microsoft started to use it, they could get sued for some abritary thing that violates someones patent.  Then Microsoft might lose because they are an "evil" corporation.

• Anonymous
  May 13, 2011
  @Andrew: DRM on MP3? Good joke. You don't want to be taken seriously,do you?

• Anonymous
  May 13, 2011
  Why does the following trigger a message in IE stating that scripts and ActiveX are restricted? <!DOCTYPE html> <html>   <body>      <audio src="django.mp3" autoplay>      </audio>    </body> </html>

• Anonymous
  May 13, 2011
  There are some more examples of html audio (working) on puzzles / demos on various browsers at my homepage: http://goo.gl/zI6A

• Anonymous
  May 13, 2011
  @JAB Creations: It seems you didn't get my point, I'll make it clearer: Complaining because Browser X supports Format A but not Format B, which Browser Y support, is pointless. @Andrew: You do know that MP3 and WMA and all those format can also be used without DRMs, right?

• Anonymous
  May 13, 2011
  For cross-browser compatibility I suggest audio versions encoded in both MP3 and OGG be specified, like this: <audio controls> <!-- Encoded for Internet Explorer, Google Chrome, and Safari --> <source src="audio.mp3" type="audio/mpeg"> <!-- Encoded for Firefox and Opera --> <source src="audio.ogg" type="audio/ogg"> </audio> This example can be found at mike-mcgrath.net/.../audio.html and works in IE9, Firefox 4, Chrome 11, Safari 5,& Opera 11.

• Anonymous
  May 13, 2011
  .jxr, .hdp  - Filename extension(JPEG XR) support!

• Anonymous
  May 13, 2011
  @Aethec, The fact remains, continuing support for MP3, AAC etc. allows big companies to use DRM. If MP3 is continued onto the web, it gives them the chance to drag DRM onto the web as well.

• Anonymous
  May 13, 2011
  @johnnyq3, That's the entire point of why software patents hold the web back. I suggest you read at http://endsoftpatents.org/ I'm sure that if Microsoft really wanted to implement OGG, they would just do it. The big boys at Microsoft have plenty of money behind them anyway.

• Anonymous
  May 13, 2011
  @Andrew Incorrect. Mp3 has no drm. Was viewed as unsuitable by publishers due to that.(only few formats have builting support for drm) MS might have money but they have better use then being on recieving end of patent lawsuit. Like buying Skype...

• Anonymous
  May 13, 2011
  @Andrew Heres the thing if Microsoft tries to stop patenting stuff on their own then someone can swoop in and patent it.  RIM did the same thing to MS with Exchange related tech.  And now RIM tried to sue MS for Exchange related patents.

• Anonymous
  May 14, 2011
  How to get ID3 info.Such as artist,lrc and so on?

• Anonymous
  May 14, 2011
  @johnnyq3, So according to you software patents are a problem but there's nothing we can do to stop it. In other words, we allow Microsoft to further infect the web with patented formats, and that's that. I mean sure, why not?

• Anonymous
  May 14, 2011
  @Andrew: They, just like many other companies, have no choice. Since that stupid patent system* is not going to disappear any time soon, they have to patent everything they can to avoid paying royalties to patent trolls such as Acacia Research. *I'm not saying patents are stupid ; the US patent system, however, is stupid.

• Anonymous
  May 14, 2011
  I've been commenting on this blog since day 1 and I really don't appreciate it when the IE team makes "elephant in the room" blog posts. How can you guys talk about the audio tag and not explicitly mention Ogg? WebM. Ogg Vorbis. WebGL. Not scoring half of what other browsers score on HTML5Test. Are these things even on your radar? Are you going to let standard bureaucracy delay things another 3 or 4 years? You guys always dodge tough questions. I'd appreciate nebulous answers over silence.

• Anonymous
  May 15, 2011
  @Kitsune: Please give a reference in the HTML5 standard that stipulates that the formats WebM, Ogg Vorbis and WebGL must be included to meet that standard. Don't let yourself be misguided just because a few people say it's a good test because it trounces IE. The distribution of points in this test was specifically weighed to do that (in my not so humble opinion.) rant: html5test is one of those "test"-sites that attempt to give a rating to standards support. The test itself just asks the browser "do you implement ...?" If the browser responds "probably", or even "maybe" then that's another point in the bag. It doesn't test the implementation or the completeness of the feature. It also gives multiple points for the same feature. See WebM support both getting points in the audio and video section.

• Anonymous
  May 15, 2011
  @alvatrus The HTML5 standard originally recommended OGG (I believe) but changed it to allow any format. However MP3 and WMA aren't exactly open for any web browser to implement: WMA - a proprietry standard which contains DRM. MP3 - no DRM (I got confused with AAC) which is patented, but supposedly an "open" standard. Note that this format actually isn't open for anyone to implement. MP4/AAC - DRM and patents. Even worse than the above two formats. OGG Vorbis - an open standard, no DRM, no patents. HTML5 is supposed to be "open". Even though MP3 and WMA are "conforming" to the specification IE is still less open. So in my eyes, and in the eyes of many others (I'm sure), IE isn't truly conforming to HTML5 because it isn't using open standards.

• Anonymous
  May 15, 2011
  Native MIDI support would be great for games!

• Anonymous
  May 15, 2011
  <bgsound> still supports MIDI, AFAIK. :)

• Anonymous
  May 15, 2011
  @PatriotB: But only through plugins, right? It's a shame as MIDI files are the like the equivalent of vector images for sound.

• Anonymous
  May 15, 2011
  @Spinoza: No plugins required - native support. At least for IE.

• Anonymous
  May 15, 2011
  IE Team This isn't anything big and stuff but all i want to see if IE10 is the tabs toolbar to become more round like. cause the square shape of it looks very odd. plus round looks nice

• Anonymous
  May 16, 2011
  var audio = document.createElement("audio"); if (audio != null && audio.canPlayType && audio.canPlayType("audio/mpeg")) { audio.src = "audio/sample.mp3"; audio.play(); } This is working great for me :)

• Anonymous
  May 16, 2011
  @All OGG fans: Sadly, there is no patent-free standard. Sorry. And WMA/AAC can be without DRM, too.

• Anonymous
  May 16, 2011
  html5test.com/results.html Microsoft Internet Explorer 9         -  130 Microsoft Internet Explorer 10 PP 1   -   130 unlocking what?? No change from 9 to 10

• Anonymous
  May 16, 2011
  @Aethec I don't care if WMA/AAC music can be played without DRM. The fact that DRM is supported is the issue. Because they are supported, websites can easily force DRM onto their users anytime they want. Oh, and by the way, OGG is a patent-free standard. I don't see anything on the web that suggests otherwise. :/

• Anonymous
  May 16, 2011
  The comment has been removed

• Anonymous
  May 16, 2011
  "So if a website want to use DRM, they should not be able to do it?" If a website uses DRM, then it may only be able to be viewed by a specific browser/OS. I know for a fact that my WMA files which I bought from NineMSN music a few years back could only be played with WMP, and are impossible to play now. Imagine what this will be like for web browsers? "Sorry, you can't view this video because you aren't running Windows. XYZ browser requires Microsoft Windows to view this content."

• Anonymous
  May 16, 2011
  The comment has been removed

• Anonymous
  May 16, 2011
  The comment has been removed

• Anonymous
  May 16, 2011
  @Andrew: The fact Xiph says so does not make it so. Don't you think Microsoft was sure that Word didn't infringe i4i's patents, too?

• Anonymous
  May 16, 2011
  @Andrew OGG Vorbis is not a standard at all. It is a proprietary format owned by Xiph. It is free to use and has open specs.

• Anonymous
  May 17, 2011
  The comment has been removed

• Anonymous
  May 17, 2011
  @Eric Read tools.ietf.org/.../rfc3533 @Klimax I don't see any reason to give up. Lots of people don't have Silverlight, and I already use Flashblock (which is incredibly useful).

• Anonymous
  May 17, 2011
  @Andrew The Ogg container has been submitted as a possible standard (it's not one yet) but the Vorbis codec is an entirely different matter.

• Anonymous
  May 17, 2011
  The comment has been removed

• Anonymous
  May 19, 2011
  Thank you for supporting popular formates like Mpeg-4 and Mpeg-1.

• Anonymous
  May 23, 2011
  The comment has been removed