Understanding Differences in Hardware Acceleration through Paintball

While most modern browsers now support hardware acceleration, the quality of implementations
varies greatly between browsers. Hardware acceleration is more than just a checkmark.
How a browser chooses to enable hardware acceleration has a direct impact on the performance
and user experience of Web applications.

To highlight the quality and performance differences in hardware acceleration, let's look closely at the
Paintball test drive, released with
IE 10 Platform Preview 1. Paintball uses HTML5 Canvas to simulate shooting
paintballs against a wall. Paintball demonstrates how quickly images compose an entire scene.
Many other HTML5 Canvas performance examples on the web just measure how quickly the same image can be drawn on
the screen.

Paintball draws an entire scene as quickly as possible. At its end, you'll see two numbers. The first is how long drawing
the entire scene took in seconds; the second is the number of paintballs fired per minute.
In this test, more paintballs in a shorter time is better.

Screen shot of Paintball demo rendered in IE10 Platform Preview 1

How Paintball Works

We wanted Paintball to be a general purpose Web application for painting any phrase with any color of paintball. To this end, Paintball programmatically
generates paint splatters at runtime based on the specified RGB values. Programmatically
manipulating images at runtime is a common scenario for emerging HTML5 Web applications.

Paintball starts by drawing a solid rectangle which is the color of the paint splatter.

Paintball uses a sprite sheet to hold the frames of the splatter animation:

Paintball decides how large the resulting splatter should be, and draws a frame
from the sprite sheet using the destination-in composition mode. This causes
the pixels being drawn to mask the current contents of the canvas.

By generating the graphics on the client, Web developers are able to build and download fewer images. This is a great approach for programmatically changing the color of an image.

The paintballs are created in a canvas which is not visible on the screen, and then
drawn to the primary display canvas. Using a secondary, off-screen canvas to compose the effects improves performance quite a bit, in all browsers, by using composition operations
only while the animation is in progress. Once the animation has reached its final frame (paint fully
splattered), we can use a cached view of the secondary canvas when drawing the full scene.

Quality Differences

Paintball uses the same HTML5 markup for all browsers as specified by the HTML5
Canvas specification. Although all modern browsers now support these API’s, the quality
differences are extremely noticeable. Here’s a visual comparison of the rendering
differences:

You'll notice the following differences:

• Chrome 12 does not draw the paint splatters correctly because
  Chrome does not implement the HTML5 Canvas composition modes according to the
  HTML5 specification.
• Firefox 4 renders the scene correctly.
• Internet Explorer 9 and Internet Explorer 10 render the scene correctly.
• Safari 5 shares the paint splatters rendering artifacts with Chrome. Additionally,
  Safari does not yet support WOFF fonts.

These differences capture the kinds of problems that developers face building sites that look and work consistently across browsers. Developers want to use the same markup
and experience interoperable results. While these bugs might seem minor (and shouldn't be hard to fix), they represent the quality issues across browsers that make developers resort to
plug-ins or lower their expectations for sites.

Performance Differences

The performance differences between implementations are even more striking. For our testing we used the same mid-range configuration that
we
have used throughout the last year. Here’s how quickly these modern browsers,
which all have some degree of hardware acceleration, compose the scene (on Windows 7):

	Chrome 12	Firefox 4	IE 10	Safari 5
Seconds (lower is better)	516.14	100.17	11.35	68.11
Paintballs per minute (higher is better)	10.11	52.11	459.95	76.64

What’s Happening on the Hardware?

Let’s look at detailed performance traces to see how the different browsers use the
CPU and GPU when animating a single paintball shooting and splattering onto the
screen. The below results were generated using the
Windows Performance Toolkit.

With Chrome 12 you’ll notice that it takes 3.7 seconds to shoot a single paintball.
We can see that when the paintball first appears on the screen as a ball, there
are lots of updates with moderate GPU and CPU utilization. Chrome is able to maintain
almost real-time animation. Once the splatter effect (described above) starts to
render, we see GPU and CPU utilization spike for several seconds. During this time,
there are no screen updates, so the demo appears to freeze. Finally, the browser
catches up and the scene finishes rendering.

Firefox 4 handles this scene quite well in 0.550 seconds. CPU and GPU utilization
is fairly steady throughout the animation although the CPU activity does increase
as the splatter effect is rendered. Screen updates occur predictably, rendering
19 frames during the rendering of the scene.

IE 10 renders the scene quite smoothly, in the shortest amount of time (0.475 seconds)
and with the most frames. From the below graph, you can see how CPU and GPU utilization
are lower than with Firefox, and the screen updates occur more frequently and at
more regular intervals.

Performance in Safari 5 can be misleading from the CPU/GPU measurement perspective.
While the GPU utilization is lower, and that screen updates occur frequently at
regular intervals, nothing appears to be happening on the screen. That’s because
Safari is telling the GPU to paint the same image over and over again. Safari only
draws a few images to the screen and then jumps to the final frame without any animation
occurring. Safari feels hung and unresponsive during this time.

The graphs above show a single paintball splatter – a small slice in the overall
work done to render the entire Paintball demo. While most browsers can handle a
single paintball well, many of the browsers have a difficult time scaling to dozens
of paintballs. For example, Firefox renders a single splatter fairly well (almost
as fast as IE 10). However, when faced with a higher load in the full demo experience,
performance in Firefox slips to about 20% of Internet Explorer 10.

Here’s the total CPU time, GPU time, and visual updates across browsers.

Summary

Browsers with full hardware acceleration and fully interoperable implemenations of HTML 5 features deliver a better experience for
Web developers and end users.

We hope this look into the internals of Paintball will help you better understand
the different quality and performance characteristics across browsers, particularly
in relation to HTML5 Canvas and composition modes. It’s an exciting time to be developing
graphically rich Web experiences!

—Seth McLaughlin, Program Manager, Internet Explorer Performance Team

Comments

• Anonymous
  April 26, 2011
  If you're comparing browser performance against IE10, why aren't you using Mozilla Aurora, instead of Firefox 4, as it contains a number of hardware acceleration related fixes? Somehow I doubt you're unaware of its existence, given the obvious attention you pay to what your competitors are doing, so why was it excluded? Perhaps because the results would not have been as favourable to you?

• Anonymous
  April 26, 2011
  @Jon: Most likely this blog post was written before Mozilla had released any Aurora builds since testing things like this probably takes a bit more time than just one day.

• Anonymous
  April 26, 2011
  @Jon: Our goal was to compare the fastest available builds of each browser. In Firefox Aurora, we saw lower scores than Firefox 4 final (about ~164 seconds in FFAurora vs. ~100 seconds in FF4). This is why we used the FF4 numbers. Thanks!

• Anonymous
  April 26, 2011
  Jon just got pwned...

• Anonymous
  April 26, 2011
  Lol, Jon just got so pwned.

• Anonymous
  April 26, 2011
  Creating a "benchmark" that favors your own specific implementation isn't hard.  That's why performance tests are usually performed by an impartial third party. Microsoft, please stop trolling. Or don't, actually.  Chrome and Firefox developers will soon enough make the tweaks necessary to run this test just as fast, and that's probably not a bad thing.

• Anonymous
  April 26, 2011
  Firefox renders the scene differently but still correctly?

• Anonymous
  April 26, 2011
  Firefox is struggling with panning and zooming and this is what lets it down in the test ... this is a weakness so now we can work on it and make it better. Top marks to the IE devs ... more releases = more cakes ;o)

• Anonymous
  April 26, 2011
  Other browsers will make tweaks to run this fast? Ha! Right. As much as I loathe the continued presence of IE6 and even IE7. As much as I don't like IE8. As much as IE9 wasn't quite catch up in terms of features (IE10 may well be though), if there's one thing that I've not seen is anyone demonstrate a browser acceleration technology as rapid as Microsoft's. I think what they're saying about working as directly as possible with the hardware makes perfect sense. I don't think we'll ever see Chrome get as fast because of it's compatibility layering in the API and Chrome is my regular browser (make IE10 look pretty with tabs and I might be tempted to switch). Firefox has decided to jump on board the Direct2D API and on Windows this really shows. Firefox's hardware acceleration is way better than Chrome's, but Firefox is still behind Microsoft. I don't expect to see anything beat IE10, other browsers are too focussed on XP and other platform compatibility. A design approach has to compromise somewhere. IE9+ compromised with legacy OS compatibility. I know for sure that as a web dev, it's not IE that will be slowing my sites down in the future (in terms of graphics anyway). Maybe some dedicated linuxy-browser with a similar direct hardware API to Direct2D could cut the mustard. Give it 5 years or so, you never know.

• Anonymous
  April 26, 2011
  @Jon just got his player card revoked. Better question for you guys at Microsoft. When will we see something like the requestAnimationFrame() so animations wont reduce the browsers speed when tabs are changed? Graphically intense sites shouldn't gobble up resources if they're not being actively viewed, right? Keep up the good work.

• Anonymous
  April 26, 2011
  These benchmarks are (as mentioned above) a classic case of being targeted specifically for the strengths of a particularly browser implementation. You only need to run this * independent* HTML5 canvas benchmark to see: http://bit.ly/astbench - IE9 (and IE10) perform amazingly well on the pure bitmap tests (like above) but are very poor at shadows and sub-pixel primitive rendering i.e. lines and polygons - Chrome is much faster at those tests. It would be nice if Microsoft released tests that actually covered all the features people are using in canvas, not just the subset they know are well GPU accelerated.

• Anonymous
  April 26, 2011
  The comment has been removed

• Anonymous
  April 26, 2011
  As with FishIE, I imagine Firefox would significantly outperform IE10 with a WebGL implementation (muizelaar.blogspot.com/.../drawing-sprites-canvas-2d-vs-webgl.html). I'd be interested to hear what the roadmap is for adding WebGL support to IE. And while we're talking about roadmaps, what's the schedule for Microsoft to join the WebM CCL (http://www.webm-ccl.org/)?

• Anonymous
  April 26, 2011
  @Another Another Jon: All of the browser makers use benchmarks that work best for their browsers. Kraken for Mozilla, V8 for Chrome for example. You could ask them to stop trolling too.

• Anonymous
  April 26, 2011
  @Matthew: Unless GPU produces care more about Linux there won't be much going on. @KurtDame: I would guess that the performance numbers don't differ that much in WebGL (be it that they run on top of the same WebGL platform, e.g. OpenGL, ANGLE,  Mesa3D). On the other hand should MS really plan to implement WebGL(which I somehow doubt) then they would surely create their own WebGL-to-DirectX mapper out of their not-implemented-here attitude.

• Anonymous
  April 26, 2011
  Comparing this post to the canvas-vs-svg one makes me sad. I'll try to enunciate why, and sorry if it comes out confusing. What's your message to (your) developers? Is it: IE on windows is the fastest, so make apps to take advantage of that? Is it: Browser speed matters greatly to DEVELOPERS? Then all I have to say is, poor developers, they are being mistreated again. You are (yet again) not giving them the right tools to succeed in today's world. (akin to what Microsoft did to Silverlight developers, who found the rug pulled from under them) I really really don't care, as i really find this blog mostly irrelevant (!), but I think your message should be: we like standards, so should you. Make your sites cross-platform and cross-browsers compatible. Learn about standards from whichever source you can find: Yahoo Developer Network (Doug Crockford), Google I/O videos, Mozilla's site etc. Make sure your site works from all browsers/platforms. IETeam you should ask yourself, what has Microsoft/IE contribute to the web? is in the field of H/W acceleration?? Is it in the form of Win7 pinned sites?? It seems to me that Microsoft found itself disrupted very badly, and is struggling to define itself and it's mission (other then to make money). com gmail meni.taub

• Anonymous
  April 26, 2011
  yeah yeah those fancy browsers can paint stuff really fast. Now give me something I can use in my real job like the new HTML5 inputs. I would also like an option to make the text rendering the same as IE8 (or compatibility view for that matter).

• Anonymous
  April 26, 2011
  The browser comparison is never a fair one because, to use the same markup everywhere all people in the world should have an updated browser. This Cycle is like 6 years for Internet Explorer (because some people still use IE 6 these days) and all the other browsers will be auto-updated in time. So Internet Explorer will allways slowdown the web, even when there BETA version has much better support of standerds, etc..

• Anonymous
  April 26, 2011
  The comment has been removed

• Anonymous
  April 26, 2011
  Laughing Out Loud !!!  Great idea, btw. Can't phathom (insert spelling-checker here - I'm not a native English speaker :-( ) the avalanche of comments that would attract, though. And while we're at it, can I ask all the WebM affectionados for a small token of compensation for the $5000 equipment (cams, videos, software, etc.) I need to replace with "open" variants that support WebM instead/alongside H264?

• Anonymous
  April 26, 2011
  You gotta love open-source fanboys...when Microsoft are good at their own benchmarks, it's because they wrote them, but when Firefox is good at Kraken and other Mozilla benchmarks, it's because FF is so great... ...by the way, please don't support WebGL. Write specs for a 3D Canvas API if needed, but don't support WebGL. Unless all other browsers start supporting -ms-filter.

• Anonymous
  April 26, 2011
  Eh.  I love the product, I'd love other engines to be every bit as good at graphics, but I don't think catfights with other vendors on the official blog is either classy or useful.  Unsubscribed in my RSS reader, but I'll check the IE10 release notes when it comes out.  I'm told that everything  is merely activity, not progress, until it's "site-ready" anyway. ;)

• Anonymous
  April 26, 2011
  The comment has been removed

• Anonymous
  April 26, 2011
  The comment has been removed

• Anonymous
  April 27, 2011
  @hAl, my T-Mobile G2x is capable of 1080p playback. :)

• Anonymous
  April 27, 2011
  The comment has been removed

• Anonymous
  April 28, 2011
  The comment has been removed

• Anonymous
  May 04, 2011
  Please consider UI performance in IE10, especially on low end ULV machines, where Chrome feels much faster. For example time tab openings, closings. Also, fix horizontal scrolling speed in zoomed in images (small increments atm) and also consider a vertically larger address bar, another advantage of Chrome once you get used to it. Also, the tiles in the home page sometimes pick up the 2nd or 3rd page of a blog.

• Anonymous
  May 05, 2011
  Also, if the net connection is bad (icon says ok but no packets get thru) then there is a lenghty delay on loading - 5-6 seconds. This happens on bad wifi access points and it appears to the user as an IE flaw, please fix.

• Anonymous
  May 05, 2011
  @zzz: It makes less sense for code, ie. fixed-width font you rarely print, now for the web it makes a lot of sense like explained numerous times and demonstrated in some IE Testdrive demos.