A GPU-Powered HTML5 Flickr Photo Viewer

With Internet Explorer 9, developers can build new classes of HTML5 applications that were previously not possible. We’re having fun building sample web applications that provide a glimpse into the types of experiences that hardware acceleration provides. In this post, we take a closer look at Flickr Explorer, one of the samples that we released alongside IE9 Platform Preview #2.

Flickr Explorer is written using standard HTML, CSS and JavaScript. It uses AJAX to asynchronously search Flickr based on your search terms and display image results through an interactive and visually compelling experience. You can zoom in, pan around, and open each photo’s Flickr page with by control+clicking.

With Internet Explorer 9, Flickr Explorer is generally able to maintain a near real-time responsiveness of 52fps (52 frames per second). In contrast, other browsers struggle to maintain 4-8fps, which is barely 15% the performance that Internet Explorer 9 provides in this particular scenario.

As we discussed last month in a post on the Flying Images demo, we redesigned the core of Internet Explorer 9 to be fully hardware accelerated. Internet Explorer 9’s display rendering subsystem uses the GPU for all graphics and text on web pages. Internet Explorer 9 moves graphics work that has traditionally occurred on the general purpose CPU to faster, more specialized hardware. Internet Explorer 9’s JavaScript engine takes advantage of multiple-processor cores to background compile JavaScript into machine code. Internet Explorer 9 uses modern processor instruction sets across the entire product. Taken together, these changes enable computations to occur faster and in parallel, freeing the CPU to spend time performing other operations.

Let’s take a closer look at the CPU and GPU activity graphs collected while running the Flickr Explorer sample to see more hardware acceleration in action. These traces were captured while zooming in on a selected Flickr photo, and use the same machine and methodology that we previously discussed.

First, results from Internet Explorer 8 are shown below. Like we saw with the earlier Flying Images demo, Internet Explorer 8 utilizes an entire CPU core (50% of this dual core machine) to animate the images on screen. Internet Explorer 8 does not utilize the GPU in this scenario.

Internet Explorer 8 can barely make one screen update every 0.253 seconds, which results in roughly 3.9 frames per second (FPS). This is clearly a very poor experience for the user.

Next, let’s check out how the new Chrome 5 beta handles this task. As you can see below, it doesn’t fare much better than Internet Explorer 8. Chrome is able to update the screen every 0.222 seconds, achieving a frame rate of about 4.5 FPS. Again, this results in a very choppy and undesirable experience. Chrome also does not utilize the GPU in this scenario.

Safari 4 handles the demo similarly to Chrome, earning a slightly lower frame rate of 4.2 FPS. Again with Safari, the GPU remains unused while the CPU remains a bottleneck.

Next, let’s take a look at Firefox. We used the most recently Firefox nightly build, Minefield 3.7a5, for this analysis. Like each other browser tested, we ran Minefield in its default configuration (which means hardware rendering with D2D was not enabled). We will post comparisons with Firefox’s hardware acceleration when it’s on by default in their beta.

Firefox handles the demo much better than Chrome and Safari. The screen gets updated roughly once every 0.12 seconds, which results in a frame rate of about 8.3 FPS. While this is nearly double the score of Chrome, it is still quite slow and barely usable.

Finally, the results from Internet Explorer 9 Platform Preview 2 are shown below.  We can see that, unlike with the other browsers, the CPU handles this task with ease and has periods of frequent rest where Internet Explorer and your applications can performance additional operations, while the GPU renders Flickr Explorer to the screen.

Internet Explorer 9 is able to update the screen once every 0.019 seconds on this mid-range configuration, which equals a frame rate of about 52 FPS.

When you play with the demo, the difference between Internet Explorer 9 and other browsers is crystal clear. While you can easily zoom in, out, and pan through the Flickr photos with Internet Explorer 9, it’s difficult to do this with the other browsers.

We’d like to extend a special thanks to the Flickr team for building such a simple, easy to consume services API. We were able to put this sample together in a couple of days using the same APIs that web developers program against today. The only difference is that we have higher expectations for the types of application that you can build through standard markup and these APIs.

We’re excited to see what new experiences web developers will create with IE9’s hardware acceleration!

Seth McLaughlin
Program Manager for IE Performance

Comments

• Anonymous
  June 03, 2010
  I am getting excited to start seeing GPU powered browsers. Good Job. Just might get me to switch back, maybe.

• Anonymous
  June 03, 2010
  Good Job.

• Anonymous
  June 03, 2010
  I understand why you didn't include the Opera results, since it even without any hw acceleration activated it is equally fast as IE9 PP #2 (50+ fps with version 10.53 on Windows 7 with a modest tripple core processor). My Firefox 3.6.3 did around 16 fps, while the non-beta Chrome 5 wasn't able to run it (no pictures was shown).

• Anonymous
  June 03, 2010
  Browser tests are interesting, because small bottlenecks can make a huge difference. They can show us with which problem people creating rich(er) web applications will be confronted in the future. What I would be interested in, is to know why IE8 and WebKit-based browsers like Safari and Chrome perform so horrendous at the Browser Flip test while current versions of Opera and Firefox fare quite well. On the other hand there is this "Chrome experiment" at mrdoob.com/.../depth_of_field which runs quite nicely on Chrome and Opera while not so much under Firefox (sadly, it won't run under IE9 properly even when spoofing the user agent [I tested it with the first preview version]).

• Anonymous
  June 03, 2010
  Nice, and I certainly applaud the IE team's dedication to performance and making DOM-based animation "just work" without special work by authors or new APIs. It's also true, however, that you can do this stuff efficiently with HTML and CSS on Safari, with CSS transforms and transitions.  www.satine.org/.../snowstack.html is an example; it even has a fun 3D perspective effect on my computer. Does all this perhaps mean we'll see www.w3.org/.../css3-transitions or even www.w3.org/.../css3-3d-transforms in a future IE?  I'm sure the IE team can only do a certain number of revolutionary changes to IE at once, but I'll be a happy man when/if I see a Microsoftie join as an editor of one of those documents. :)

• Anonymous
  June 03, 2010
  ship this puppy.

• Anonymous
  June 03, 2010
  The comment has been removed

• Anonymous
  June 03, 2010
  C'mon, where is Opera? Chrome and Safari have the same rendering engine anyway.

• Anonymous
  June 03, 2010
  Until you compare HA browsers with other HA browsers these comparisons remain very uninteresting.

• Anonymous
  June 03, 2010
  @Boring It's interesting that other browsers don't have HA enabled by default. Might indicate it's buggy in those browsers. Most end users will run the browser with its default config. So it's fairest to test them in their default config.

• Anonymous
  June 03, 2010
  And Opera is obviously out, because the test runs at 20fps on my netbook...

• Anonymous
  June 03, 2010
  Or Opera is not included because it has <3% of the market share.  Although, limiting comparisons to the top 3 competitors has the added bonus of making IE9 looking even more impressive.

• Anonymous
  June 03, 2010
  Why did't you turn Direct2D on for Firefox 3.7a? It's just two clicks. Firefox should be faster than IE9 then. Oh, maybe that was the reason, never mind.

• Anonymous
  June 03, 2010
  Here comes the web-top!

• Anonymous
  June 03, 2010
  And yet, small fonts look horrible in IE9 by comparison with IE8/Cleartype/GDI, and there are serious problems with highlighted text selection. So before you start crowing, please make the result look equivalently good.

• Anonymous
  June 03, 2010
  @Lari The test is meant to test default settings on a browser.  GPU acceleration is default on IE9, it isn't true with FF3.7

• Anonymous
  June 03, 2010
  Yeah, it only takes two clicks in Firefox to get a browser that looks ugly, renders poorly, and crashes. No thanks... I'll wait for a stable release.

• Anonymous
  June 03, 2010
  So how long until we get hardware acceleration in Silverlight? ok ok, I'll stop :P I know it's a completely different team. Great work! I've got one question though, what sort/level of GPU are you testing? What are you expecting to work with? My graphics card, Geforce 6800GS gets a 5.9 on the WEI scale, but it's only dx9, although it has a WDDM driver... so.. would that be expected to work, work well?

• Anonymous
  June 03, 2010
  Oh and I'm assuming that we'll have to structure our webapps in a certain way to get most out of IE9. Will MSDN (or something better) start including performance information?

• Anonymous
  June 03, 2010
  Where is Opera? FPS is over 50 all the time WITHOUT hardware acceleration.

• Anonymous
  June 03, 2010
  Microsoft can't mention Opera. They were shameless enough to name their JS engine Chakra. Gee, wonder where they got THAT from.

• Anonymous
  June 03, 2010
  @fowl: Generally speaking, the goal is "same markup" so web developers will have a great experience without any particular "structure" for IE. In terms of the graphics card, 5.9 is a very good score compared to what we test with. My main machine (Lenovo x200) gets a 3.4 for "Gaming Graphics" and 4.1 for Aero.

• Anonymous
  June 03, 2010
  So this means DX was used EVERYWHERE? So please support direct manipulation on TEXTURE BITMAP DATA and USER-DEFINED SHADERS. Shaders can be provided in a seperate file and compiled by IE. for example: <link rel="-ms-shader" name="SHADER1" href="file.effect" /> Then,in CSS: element {  shader : "SHADER1(),SHADER2(5)" } scripting: if(element.msSh aders){    element.msShaders[0].args[0] = 3 } Texture manipulation: var tx = new Texture(); var tx2 = imgElement.msToTexture(); tx.setPenStyle(0xff0000,1,0); // red, width:1, solid tx.renderElements(Texture.POLYLINE, [0,0,0,1,1,1,1,0]) ... element.setTexture(tx) It may be cool!

• Anonymous
  June 03, 2010
  @EricLaw: So you must implement a method for developers to draw images -- i mean bitmaps -- on IE. In other browsers, it's called <canvas>

• Anonymous
  June 03, 2010
  The comment has been removed

• Anonymous
  June 03, 2010
  Lots of dazzle to divert us from the main show, Microsoft? Trying to make us forget all the things IE9 won't do? It's still over a year away and it won't handle properties all the others do today. This thing is dead. Give it up.

• Anonymous
  June 03, 2010
  We need "Web Worker" also for the more fast performance. It is possible to use the multi-core cpu and multi threads.

• Anonymous
  June 03, 2010
  The comment has been removed

• Anonymous
  June 03, 2010
  @Constantine Opera is not using GPU/hardware acceleration yet. Just thought you should know. It's pure software rendering :)

• Anonymous
  June 03, 2010
  weird .. Opera 10.53 60fps :D when mouse gone rampage i got 42 fps :D

• Anonymous
  June 03, 2010
  OK, cool.  I am ready to come back to IE. Now, will I get it in Q3?  Q4?  Do I have to wait until 2011?  Chrome won't take that long.

• Anonymous
  June 03, 2010
  I want Slovak IE9 version in Windows Update this year!!!

• Anonymous
  June 04, 2010
  The comment has been removed

• Anonymous
  June 04, 2010
  The comment has been removed

• Anonymous
  June 04, 2010
  @Brian - yea, I know ;-). I was hoping that if Microsoft weighed in with "well, we're gonna implement gradients in IE9", that would give a kick in the pants to the working group to settle on a final syntax. The other working drafts that I mention do seem to be pretty stable between Webkit & Gecko, so I'd hope that Microsoft would feel like they could put those CSS definitions behind their vendor-specific '-ms-' prefix (like Webkit & Mozilla have done) and go ahead and implement them. In fact, Apple released a new page today that shows off some of the new capabilities of "HTML 5", such as it is ;-). I'd love to see this stuff running in IE9! :-) http://www.apple.com/html5/ Cheers,

• Bill

• Anonymous
  June 04, 2010
  The comment has been removed

• Anonymous
  June 04, 2010
  @avokicchi, Even if they told us what parts would be supported, it wouldn't stop the flurry of 'you MUST support feature y' comments!

• Anonymous
  June 04, 2010
  @David Simply use devtools or fiddler to change browser detection... But I agree,it is bad idea on their part.

• Anonymous
  June 04, 2010
  @David: "Unfortunately Apple is using browser detection to stop non-webkit based browsers to access those demos." Haha, boasting about how HTML5 is a standard and all that, yet blocking other browsers? How ironic.

• Anonymous
  June 04, 2010
  @Stifu: If you look at the source code it seems like they have originally intended to support other browsers as well (see for example the prefix list in "showcase.js").

• Anonymous
  June 04, 2010
  Great job! It seems that the IE9 team is dead serious about HTML5 video - GPU hardware acceleration would be most useful for HTML5 videos. C’mon, how often does one see a fanciful and confusing user interface? haha ;)

• Anonymous
  June 04, 2010
  The flickr photo viewer gets stuck in IE9 when I enter a search phrase.

• Anonymous
  June 04, 2010
  I do get 60fps with Safari 4 on the Mac.

• Anonymous
  June 05, 2010
  I don't use Opera normally except to test websites for clients, but I tested Opera based on the comments, and shame on whoever decided to leave it out.

• Anonymous
  June 05, 2010
  Yes, shame on them for only testing on Windows and on browsers that a non-trivial number of people use! They should waste their time testing on platforms they don't compete on and against browsers that have virtually no users.

• Anonymous
  June 05, 2010
  I'm really excited about IE 9, Can't wait for new build (hoping this time it will be with UI). Keep up great work!

• Anonymous
  June 05, 2010
  The comment has been removed

• Anonymous
  June 05, 2010
  In my own tests of Flickr Explorer, Opera performed the best without sacrificing image quality whereas IE9 came second on my Intel X3100 graphics laptop with Windows 7 32-bit.

• Anonymous
  June 05, 2010
  Leaving out Opera is ridiculous. I just ran the Flickr Explorer demo on my Asus 1005 HA netbook and I got 30-50 fps under Opera 10.53. Shame on you, Microsoft :(

• Anonymous
  June 05, 2010
  Maybe they left out Opera due to Opera's actions against MS.

• Anonymous
  June 05, 2010
  Microsoft feel ashamed because Opera is almost winning in term of Hardware Acceleration. Come on Microsoft, don't miss out Opera test.. show us your ball!!

• Anonymous
  June 06, 2010
  Maybe they left Opera out because... the set of people who actually run Opera is microscopic? And they all seem to hang out on this blog to say "why are you excluding us?"

• Anonymous
  June 06, 2010
  I remember they showed Opera in another one of these comparisons and had to explain how Opera basically "cheated" by lowering the image quality.  Maybe it's something similar here and they didn't feel like explaining it for the minority of people who care.

• Anonymous
  June 06, 2010
  @Prior Semblance: the 'cheater' was Firefox, which used (still does?) point sampling instead of filtering on CSS images, making resizing images much faster with lower visual quality. Opera's problem is not one of market share: it's that it settled, AFAICT, on an older form of hardware acceleration: video acceleration. In short, Presto computes the page in central RAM using the CPU alone, and then blits it to screen using a video output - which is WAY faster than GDI (unaccelerated Chrome and Firefox, IE 8) and compatible with Windows 2000/XP (unlike IE 9's Direct2D). Which may demonstrate that using derelict DirectDraw over brand-new-but-gotta-ditch-XP-to-get-it Direct2D might have been sufficient for a browser...

• Anonymous
  June 07, 2010
  @Mitch74: To be fair, Firefox lets you control the sampling mode through proprietary CSS.

• Anonymous
  June 07, 2010
  The comment has been removed

• Anonymous
  June 07, 2010
  The comment has been removed

• Anonymous
  June 07, 2010
  The comment has been removed

• Anonymous
  June 07, 2010
  http://www.apple.com/safari/ Safari 5 released, embarasses Microsoft's outdated, insecure browser in speed.

• Anonymous
  June 07, 2010
  The comment has been removed

• Anonymous
  June 07, 2010
  In terms of security of browsers on Windows (vuln rate, architecture, etc), it looks like this: Chrome > IE > Firefox > Safari > Opera So they've got quite a ways to go there too.

• Anonymous
  June 08, 2010
  @Matt: I wasn't aware Chrome had more vuln than IE? I don't have more numbers but Secunia, for one, shows less vuln and less open exploits for Chrome 4.x (zero for 5.x) than IE 8.x? Could you link me? I'd be very interested in reading more about that. Thank you.

• Anonymous
  June 08, 2010
  The comment has been removed

• Anonymous
  June 08, 2010
  You forgot Opera 10.53. It would be tasteful to include it in these kinds of tests because it is the only other browser utilizing GPU acceleration on Windows.

• Anonymous
  June 08, 2010
  @Matt: Try to run the browser flip test several times on newer WebKit browsers, you will see that the FPS will go up (be it that the browser doesn't crash like Safari 5 did for me).

• Anonymous
  June 11, 2010
  The comment has been removed

• Anonymous
  June 12, 2010
  Mozilla and Google are still dedicated to supporting hardware acceleration of XP, so it's still faster in my and 60% of the world's eyes.

• Anonymous
  June 15, 2010
  The comment has been removed

• Anonymous
  June 15, 2010
  r414: Read the comments on that blog and you see how he lied.