of babies and bathwater

A couple of weeks ago, Mary gave us a behind-the-scenes look at talking to the press. One of the articles that came from the interview that she talks about is in APC: First details of Office:Mac.

In that interview, Mary was kind enough to give usability testing a shout-out. Here's the relevant part of that interview:

Design and usability testing on the Office 12 interface is already underway in the MacBU labs at Redmond and Cupertino, and the team has already made one trip back to the drawing board based on user feedback.

"We have usability experts and usability labs at both of our campuses, and we're spending a lot of time bringing people through for each iteration of the UI. That's part of why it's changing so much right now" explains Starman.

"We had what we thought was going to be this perfect UI solution, and the first time we put it in the labs, no-one understood it! It was so different they were completely confused! We just finished up another round of usability testing on the new UI yesterday, and the program manager said the difference is like night and day."

This is such a great example of what I do and why I do it. We are refreshing our UI (don't worry! we're not killing your toolbars!). We put in a lot of effort to design the refreshed user interface. But we're not perfect, and our original design wasn't perfect. When we took our new UI into the usability lab, it didn't do as well as we'd hoped. Users were really confused. So we went back to the drawing board.

Going back to the drawing board doesn't mean that we threw the whole thing out. There was definitely a baby in that bathwater. We had to figure out what worked in our design and what didn't. For what didn't work, we had to figure out why it didn't work. Were we totally wrong, or were we just a little bit wrong?

So we came up with some ideas for fixing the issues that we observed. For something as big as a UI refresh, this is quite a big deal. There were lots of discussions and brainstorms. We decided what we would change.  Mostly, they were pretty minor tweaks: making something look a bit more like a button, changing a name, that kind of thing.

The week that Mary and Sheridan did that interview, we were in the usability lab trying out the new design. Those little changes that we made had a huge impact. In the original study, none of the users really got it. In the second study, the difference was, as the PM that Mary mentions in the article said, like night and day.

Of course, it still wasn't perfect. It might not ever be perfect -- possibly because perfection doesn't actually exist for something that's as complex as what we're doing. So we made a couple of extra changes. Right now, I'm writing this blog post while sitting in the usability lab to see if this change gets us where we want to be. While I'm here, I'm also testing out a couple of new features that I didn't have enough time to test earlier. I'm not done with this study yet, but the preliminary data is fantastic. I think we've gotten what we wanted: a new UI that looks great and that helps our users do their jobs easier.

Before anyone asks, I can't answer specific questions about it yet. Not all of our functionality is hooked up, so it's still not finalised. And while I've been in the lab this week, I've discovered that there are other things that I want to look at. We solved the big problem, and the existence of the bigger issue hid many of these smaller ones now. I'll be back in the usability lab again to look at other related items. (Although our labs are in Redmond and Mountain View, not Cupertino.) If you'd like to join me and get a sneak peek at the new UI and features, sign up to participate in our usability tests!

Edited to fix a broken link. Thanks to the anonymous commenter for pointing that out!

Comments

• Anonymous
  October 03, 2006
  you might want to fix the link at the bottom of this article.

• Anonymous
  October 03, 2006
  She just fixed it. Thx!

• Anonymous
  October 03, 2006
  The comment has been removed

• Anonymous
  October 03, 2006
  Why must you guys reinvent the wheel? Can't you leave good enough alone? We've reported dozens of bugs over the last 5 years that still aren't fixed in Entourage & Word, yet you guys are completely overhauling an interface that mostly works just fine? The new "Track Changes" functionality, however is awesome -- how it now puts the changes into the side margin instead of in the text itself.

• Anonymous
  October 03, 2006
  John -- Big public betas require a lot of resources to manage.  We're a very small team, much much much smaller than the WinOffice team.  We don't have enough people to be able to adequately handle a big public beta.  If I recall correctly, one of the other bloggers is planning on talking about our beta programme in the future, so stay tuned for that. Bill -- We have to balance adding new features and fixing old issues.  You like a change to a particular feature, but I bet that someone else would say that we shouldn't've done that instead of fixing a bug that they've reported.  Everyone has different priorities. :) One of the reasons that we're refreshing the UI is to solve the discoverability problem: a feature isn't useful to to our users if they can't find it.  

• Anonymous
  October 03, 2006
  "Design and usability testing on the Office 12 interface is already underway in the MacBU labs at Redmond and Cupertino, and the team has already made one trip back to the drawing board based on user feedback. " Cupertino? Surely Mary did not confuse Microsoft with the Apple campus? Did the interviewer just invent that? Or have you left Mountain View? (Or maybe the interviewer thinks it's the same town?)

• Anonymous
  October 03, 2006
  The comment has been removed

• Anonymous
  October 03, 2006
  The comment has been removed

• Anonymous
  October 03, 2006
  Thanks for the reply on my beta question.  While I won't be looking out for Mac Office beta, I'll still keep my fingers crossed for a more Cocoa consistent look and feel.  The new UI on Office 2007 isn't bad (huge improvement over previous windows versions in my opinion) so dare we expect such bold steps in your own redesign?  I hope so!

• Anonymous
  October 03, 2006
  The comment has been removed

• Anonymous
  October 04, 2006
  Very much looking forward to Mac Office 12.  I love the Win Office 2007 beta, and hope you take some of those ideas.  Alternatively, toolbars like as in Apple's iWork are quite nice too.  Mac Office 2004 was the best office version back in its day too so I'm confident the team will do a great job with Office 12. Maybe not the right place, but I was wondering about the following: Fonts:  How come Office (2004, at least) seems to treat fonts differently from every other Apple program?  E.g. I installed some of the Office 2007 fonts (e.g. Cambria) on the Mac, and Office 2004 displays it as "Cambria" "Cambria Bold" "Cambria Italic" -- but isn't smart enough to choose "Cambria Bold" when I click the "Bold" button on the toolbar. Instead, it comes up with a faux bold which looks weird.  (applications which use the Font panel switch over to Cambria Bold automatically...)  Will Office 12 be smarter about fonts? Open format support:  When will we get filters for the Office 2007 file formats?  I sure hope I can open Office 2007 file formats (docx, etc)  when the Windows version is released, without having to wait for/pay for the next version of Mac office.  (NB the Windows version of 2000 and 2002/2003 can already open docx with a beta plugin by Microsoft...) And more to the point on usability testing - what about usability testing in other locales?  I could think of Asian markets e.g. China and Japan; as well as places where people read right-to-left being key places where you might want to conduct focus groups too...  (This is something Apple isn't good at either - e.g. the Address Book insists on dates that go M/D/Y even when you set the setting to D/M/Y...)

• Anonymous
  October 04, 2006
  The comment has been removed

• Anonymous
  October 04, 2006
  ""We had what we thought was going to be this perfect UI solution"This is a perfect example of the observation that MS developers sometimes don't have a clue :(

• Anonymous
  October 04, 2006
  The comment has been removed

• Anonymous
  October 04, 2006
  joecab: I pulled all the icons from my dock so it only shows running apps now.  I use Quicksilver as an app launcher which I find much handier than mousing anyway.  Makes for a much cleaner interface, espeically down on 1024x768 which I hope Office 12 will still be good for, hint hint.

• Anonymous
  October 04, 2006
  SL -- Our main facilities are in Redmond and Mountain View, and we do the bulk of our usability testing here.  However, we also travel around the US and the rest of the world to meet with our users and get additional perspectives on what we're doing.  I understand that usability people who work on our various Windows-based apps (and, of course, the Windows OS itself) talk about the Redmond Effect in terms of usability testing, and do a lot to try to ameliorate that.   We will release file format converters for Office:Mac 2004 so that you'll be able to open files created on Office 2007.   opensourcefan - Our testing centers are where MacBU itself is located (about half in Redmond, and half in Mountain View).  We do travel frequently.  If you haven't yet, signing up to be a part of our usability tests means that we might call you in the future to participate in a usability test.

• Anonymous
  October 05, 2006
  The comment has been removed

• Anonymous
  October 05, 2006
  Yep I have QS installed too (heard about it from Merlin Mann's 43 Folders) but I still haven't gotten around to really learning about it and using it properly. Sigh.

• Anonymous
  October 05, 2006
  OSfan -- I promise that I'll leave my MSFT ROX tattoo kit at home if I ever meet you. ;)

• Anonymous
  October 05, 2006
  I have no idea if anyone here isn't reading Mac Mojo , but in case not, my latest post over there is

• Anonymous
  October 05, 2006
  nadyne - cheers for the reply.  just one thing - will we also be able to  can we also save in the new docx xlsx pptx formats?  (It would help in a mixed Office 2004:mac and Office 12:mac environment to be able to standardise on/use the new formats when people haven't all upgraded...)

• Anonymous
  October 06, 2006
  Quote: "Give me multiple ways to get to functions, and let me tweak my UI to suite my style. Give me logically arrayed menus. Give me extensive use of the cntrl-click to access context-sensitive functionality if that's my style, but give me one or more ways to do it it that's not my style. How about a floating toolbar for context-sensitive functions?" Oh please don't! Give me a single easy, logical, and most of all consistent way to do each thing, and then stick to it! There is no other way that productivity can be improved. And visit humanized.com website regularly. P.S. This shows how diverse your user base can be. Oh boy, I don't envy you at all...

• Anonymous
  October 06, 2006
  SL -- regarding your question of font behaviors...  Yes, this is an area where Office 2004 doesn't get it right.  I believe Office 12 will do a better job with font variants. Also SL, regarding being able to save to the new Open XML formats, yes, the converters will let you do that.

• Anonymous
  October 07, 2006
  It will be interesting to see what you come up with for Office 12. In terms of discoverability, I have not had as much problems as on Office for Win which has been improved in version 2007. Office for Mac presently has a convenient UI through features such as the Formatting Palette.

• Anonymous
  October 10, 2006
  The comment has been removed

• Anonymous
  October 10, 2006
  Oops, in my last paragraph in my previous post, when I said "I hope Office 12 keeps the customizable UI and the context menus," I meant to say Office 2004, not Office 12. I haven't seen Office 12 yet!

• Anonymous
  October 15, 2006
  Probably I shouldn't say 'easy' at all. I meant efficient, the best and most consistent way to do stuff. Providing for any easy way (e.g. point-and-click as you state) quickly makes the need to find a faster way evident. And then you have to start thinking about how best to do your work again, when you should have done that only once. Spending a little more time at the rookie level should prove more efficent in the end. Anyway, this is more philosophical discussion than actual feedback to MBU. I understand that they cannot risk skaring off first-time users of their products, or alienating existing ones who have been accustomed to a certain way of working.