Test the Web Forward – Seattle 2013

On April 12-13, Microsoft hosted a record-setting Test the Web Forward event to advance the Web by creating interoperability tests. Dozens of volunteers from Adobe, AT&T, Blackberry, Mozilla, and many other local companies joined us at our Seattle offices to learn about Web standards testing, how to write CSS and HTML tests, as well as the tools available for test suite management. Attendees from around the country - and even Canada – contributed to create a record-breaking 514 overall new tests.

Why Tests?

The quality and correctness of different browsers’ HTML and CSS standards compliance continues to vary widely. The W3C requires independent tests of all normative requirements in a specification in order to move a W3C Web specification from a candidate recommendation to an official recommendation. These tests are used to ensure that at least two browsers fully support each normative statement. As you might imagine, creating all of these tests is daunting; HTML5 is expected to need well over 100,000 tests, to say nothing of CSS3 modules, WebApps, Media Extensions, etc. We have submitted thousands of test cases for HTML, CSS, and SVG that can be viewed at the W3C and the Internet Explorer Test Center, however more tests are still needed. These tests benefit all browsers – and ultimately the entire Web developer community – by ensuring a consistent, predictable behavior. As different browsers improve their same-markup support, we can all realize the promise of HTML5 and CSS3.

A few years ago, several members of the standards community turned to crowd-sourcing to help create new tests, this resulted in Test the Web Forward events.

With the sponsorship of major players like Microsoft, Adobe, Google, and Mozilla, the Web community has come together, running local test-writing sprints around the world—France, China, Australia, and the US. Each sprint not only generates hundreds of tests, but also engages with and educates Web developers about the specifications that make up the Web platform.

Inside the Seattle Event

Our friends at Adobe were instrumental in setting up a successful event, building upon their experience in running previous events. In Seattle we kicked off the hack-a-thon on Friday night, with inspirational and informative presentations from Mozilla's fantasai (Elika Etemad), Adobe's Rebecca Hauck, and Microsoft's Kris Krueger, explaining why we need tests, what type of tests are available, and how to create them. Here’s a quick outline:

Stand-alone tests typically rely on visual verification: if a failure condition occurs, red content will show.

Reference tests compare a test against a visual reference that has no dependency on the feature being tested. Note that the test includes a link to the reference test against which is should be compared. For example, if you wanted to test that DIVs render background colors correctly, you might make a ref test using TABLEs.

Object Model tests depend on a JavaScript test harness; they verify that the object model reflects what static style sheets specify. For instance, this CSS media query test.

These presentations were followed by 2-minute pitches from Saturday's session test leaders on why participants might want to pick a particular focus (CSS Flexbox, Pointer Events, CSS Transforms, CSS OM, Backgrounds & Borders, Exclusions, or HTML5), though session participants were free to write tests against any API or spec they felt passionate about.

Following breakfast the next morning, participants broke up into three rooms with session leaders helping out in each. Each area was staffed by experts (in addition to the speakers from the previous evening): Arron Eicholz (Microsoft, CSS); Jacob Rossi (Microsoft, Pointer Events); Sylvain Galineau (Adobe [formerly Microsoft], CSS); Alan Stearns (Adobe, CSS); Dave Methvin (President of jQuery, HTML).

The leaders instructed everyone on how to determine where tests were needed and how to create code that tested the specific assertion that we wanted to capture. Volunteers could either work on their own, work in small groups, or get 1:1 help with the experts.

When all was said and done, the sprint generated 514 submitted tests, just edging out the record set by the Paris test sprint and setting a new bar for upcoming sprints to beat. After a few celebratory drinks, the end of the night saw the raffling off of a Surface Pro which was won by a student volunteer who joined us from the University of Washington.

You Can Help Too!

In IE10, we have added support for a long list of new standard features across CSS, HTML, SVG and the DOM. We have published some of our test cases for these new features on our IE Test Center. We will be submitting more, but we still need help from the community to get the right tests written and move these specs forward.

We are excited to be part of the community working towards a more innovative and interoperable Web. We support several initiatives in this direction, like the recent donation of JavaScript documentation to Webplatform.org, or our continued efforts to simplify cross-browser testing with modern.ie. If you want to help move the Web forward, too, come and join us at one of the next Test the Web Forward events! In the meanwhile, you can learn how to contribute tests, or review existing tests online. To hear about upcoming events and to stay in touch with the rest of the Test the Web Forward community, please subscribe to our W3C mailing list: public-testtwf. If test writing sounds too intense, but you are still knowledgeable and passionate about the Web, you can get involved with the WebPlatform Docs project and help document the Web.

For more info and updates, follow our Internet Explorer developer relations handle on Twitter @IEDevChat, this initiative’s handle @testthewebfwd and in particular #testtwf.

We will keep you posted on upcoming events and we are looking forward to meeting you soon!

—John Jansen, Kris Krueger, Arron Eicholz, and Jacob Rossi – Internet Explorer

Comments

• Anonymous
  April 26, 2013
  The comment has been removed

• Anonymous
  April 26, 2013
  What is required is a that we can have simple listings of result for a testsuite with testsuite versions, browserversions and and result of the test. There is already more thant 10.000 test but we have no idea how any of the current browsers are doing on these test.

• Anonymous
  April 28, 2013
  Would there ever be a blog port here by F12 developer tools?? We have LOT to discuss with those guys!

• Anonymous
  April 28, 2013
  Correction: "F12 developer tools" team

• Anonymous
  May 06, 2013
  Could any of you elborate on how well the IE10 and or IE11 are scoring on these w3C tessuites. Is is of little use having good testsuites if there are no result for the testsuites. so why not provide us with some results figures?

• Anonymous
  May 06, 2013
  If you follow the link to the IETestCenter (samples.msdn.microsoft.com/ietestcenter) you can see the results.

• Anonymous
  May 06, 2013
  Can we get the IE11 developer tools ported back to IE9 and IE10?  After seeing the sneak peak screenshots with the DOM finally fixed properly we really need this updated on existing IE browsers so we can actually use the developer tools!

• Anonymous
  May 08, 2013
  Try out the preview version of IE11 in Windows Blue latest release! F12 dev tools are redesigned!!! 8-)

• Anonymous
  June 07, 2013
  Thanks for your feedback. We recognize that the current implementation is non-standard, and want to help you build interoperable web sites. In the example provided here, markers on a path are inheriting the fill opacity from the SVG element that references those markers, rather than the style cascade from the parent element where those markers are initially defined. You can address immediate problem of having the markers be semi-transparent by setting a fill-opacity style inline at the marker definition, which will override the fill-opacity style inherited from the referencing path. That said, it is true that IE’s implementation of <marker> style inheritance differs from other browsers. In short, <marker> style inheritance works like <use> style inheritance, in that styles inherit from the referencing elements. We initially chose this implementation because of the additional benefits it allows for developers, such as the ability to have a marker’s fill match the object to which it is attached. So, for example, you could define one marker and reference it from many different paths, and each time have the marker match the fill properties of the referenced element without any additional work. And, of course, you can fix the opacity by setting an inline style on the referenced element above. In contrast, if the marker did not inherit styles from the referencing object, it would be impossible for the marker to match the opacity of its referencing object. Hopefully this information helps answer your question. Our goal is generally to implement to the standards but in this case we felt the need to diverge in order to provide additional flexibility to developers. We appreciate community feedback on that choice. Best regards, Chris Jones Program Manager Internet Explorer