Inside Deep Thought (Why the UI, Part 6)

This is the sixth part in my weekly series of entries in which I outline some of the reasons we decided to pursue a new user interface for Office 12. You can read the last installments here: Part 1 Part 2 Part 3 Part 4 Part 5.

Microsoft is tracking your every move!

Soon after you install Office 2003 on your computer, a balloon pops up asking if you would like to "Help Make Office Better."  If you click on it, you are given the opportunity to enroll in something called the Microsoft Office Customer Experience Improvement Program.  If you opt-in, anonymous data about how you use Office are uploaded to Microsoft occasionally in the background.

If you're the curious type, you might have wondered where your data goes.  Well, today I'm here to answer the question: it goes into an Excel spreadsheet I have sitting on my desktop.

OK, back up.  Back in the olden days of designing software at Microsoft (say, pre-2003), design decisions were mostly supported by guesswork.  There's a classic Microsoft interview question (that I've never heard of anyone actually using) "How many gas stations are there in the United States?"  Many have criticized that type of question as being feckless; personally, I agree and it's not representative of how I choose to spend my interview time with a candidate.  But the rough "estimate an answer and defend it" style required to answer the gas station question was at the heart of how many design decisions used to be made at Microsoft.

Suppose you were designing the adaptive menus in Office 2000 and you wanted to know what features people use the most.  Well, you start by asking a "guru" who has worked in the product for a long time.  "Everyone uses AutoText a lot," the guru says.  The louder the "experts" are, the more their opinions count.  Then you move on to the anecdotal evidence: "I was home over Christmas, and I saw my mom using Normal View... that's probably what most beginners use."  And mix in advice from the helpful expert: "most people run multi-monitor, I heard that from the guy at Best Buy."

So much of what we did was based on feel, estimation, and guesswork.  How much that was true only became clear with the introduction of a technology called SQM (pronounced "skwim") .

SQM, which stands for "Service Quality Monitoring" is our internal name for what became known externally as the Customer Experience Improvement Program.  It works like this: Office 2003 users have the opportunity to opt-in to the program.  From these people, we collect anonymous, non-traceable data points detailing how the software is used and and on what kind of hardware.  (Of course, no personally identifiable data is collected whatsoever.)

As designers, we define data points we're interested in learning about and the software is instrumented to collect that data.  All of the incoming data is then aggregated together on a huge server where people like me use it to help drive decisions.

Hard at work in the SQM data center.

What kind of data do we collect?  We know everything from the frequency of which commands are used to the number of Outlook mail folders you have.  We know which keyboard shortcuts you use.  We know how much time you spend in the Calendar, and we know if you customize your toolbars.  In short, we collect anything we think might be interesting and useful as long as it doesn't compromise a user's privacy.

How much data have we collected?

• About 1.3 billion sessions since we shipped Office 2003 (each session contains all the data points over a certain fixed time period.)
   
• Over 352 million command bar clicks in Word over the last 90 days.

Reflected in these numbers is that we don't even retain all the data points we receive... particularly, we get so much Word and Outlook data that 70% of it is thrown away.

So, one of the biggest reasons that we decided to do the new user interface for Office 12 is simply that, for the first time, we have the data we need to make intelligent decisions.  Anything we would have done in the past would have been based more on guesswork and bias than on reality.  Data is just one input to the design process, of course, but there's something extraordinarily empowering about knowing which commands people use often and which they don't.  And knowing which commands are used in sequence with which other commands.  And which commands are used 7x more with the keyboard than with the mouse.  And how big people's screens are... and how much of the time they use Excel maximized... and how many documents they use at once... and which commands literally are never used... and which are used much more frequently by East Asian users... and on and on...

Knowledge is power.  And having that knowledge makes this the right time to reinvent the user interface of Office.

Want to guess what is the most-used command in Microsoft Word?  The top 5 commands used?  Post your guesses in a comment and I'll answer in next Monday's post.  (MVPs who saw my talk at the summit and therefore know the answers, please refrain from showing off.)

Comments

• Anonymous
  October 31, 2005
  ctrl+c/ctrl+v
• Anonymous
  October 31, 2005
  Close?
  
  ;)
• Anonymous
  October 31, 2005
  "Yes submit this crash to Microsoft and restart the app" :-)
  
  Just kidding. Sounds like a fascinating initiative and drilling around in all that data has yielded tons of new intelligence already. Nice job
• Anonymous
  October 31, 2005
  Here's what I would expect is in the top 5:
  
  Font
  Bold
  Italic
  Save
  Print
  
  I'm probably not even close... am I?
• Anonymous
  October 31, 2005
  File-Save
  File-SaveAs
  File-Print
  Those seem the most popular on my desk. :)
• Anonymous
  October 31, 2005
  File-Open
  File-Save
  File-Print
  Edit-Copy
  Edit-Paste
• Anonymous
  October 31, 2005
  The comment has been removed
• Anonymous
  October 31, 2005
  Top 5
  Save
  Paste
  Copy
  Cut
  Spell Check
• Anonymous
  October 31, 2005
  save,
  open,
  print,
  cut,
  paste,
• Anonymous
  October 31, 2005
  I'm going to go out on a limb and guess that the most often used command is Undo. After all, to do five different things, I use five different commands; to undo five different things, I use the same command five times.
• Anonymous
  October 31, 2005
  Ctrl-Z Undo has got to be one of the top 5. I'm sure that bold/italic are in there too.
• Anonymous
  October 31, 2005
  #1 Undo
  #2 Paste
  #3 Copy
  #4 Cut
  #5 Save
  
  
• Anonymous
  October 31, 2005
  Everyone's top five:
  
  Save
  Cut
  Paste
  Bold
  Underline
  
  My own top five:
  Ctrl+Z
  Ctrl+C
  Ctrl+X
  Ctrl+S
  Ctrl+Shift+S
  
• Anonymous
  October 31, 2005
  Heh, so we're playing "How many gas stations are there in the United States?" anyway.
  
  I'd say Undo, Cut, Copy, Paste, Save.
  
  Defense: These are used multiple times when editing a document. Commands like Open, Close, Exit, and Print are usually only used once per document. Bet even formatting commands like Bold and Italic beat out Open and Print.
  
• Anonymous
  October 31, 2005
  PS - I meant to justify my answers! You only OPEN a document once per visit to the document.
  
  But you will probably do lots of saving and formatting.
  
  You may only print it once or twice.
  
  Therefore, Open won't be too common. Save will be down the list, but higher up than printing...
• Anonymous
  October 31, 2005
  My thoughts are:
  
  #1 Undo (for the same reason back is #2 on browsers; would be typing if it was a command)
  #2 Paste
  #3 Save
  #4 Cut
  #5 Copy
  
  Select Text if it's considered a command would be close to number #1 if it's a command.
  
  However, if this were a trick question, then there's also the possibility of AppMove or AppRestore or anything else to do with windows management.
  
  Also prime possibilities: Accept Changes and Reject Changes.
  
• Anonymous
  October 31, 2005
  Fonts
  Copy
  Paste
  Undo
  Print
• Anonymous
  October 31, 2005
  I'm betting it's Save and Paste. Well, the former might be wishful thinking.
• Anonymous
  October 31, 2005
  Save and undo would top my list. Why? Well, aren't those the only buttons that remain on the one remaining "toolbar" in Office 12?
• Anonymous
  October 31, 2005
  I'm betting Save is very rarely used. Most end-users I've known are very hostile to the idea of saving frequently. The problem is that they have to stop and grab the mouse to do almost anything, and that's a major hassle when you're trying to get work done. Word has had auto-save since forever, hasn't it?
  
  If backspace counts as a "command", it's probably the most commonly used.
  
  The punchline of this "guess the most common command" exercise, I suspect, is going to be just how far out of touch with the common user programmers tend to be. That's an important point. It can't be over-stressed.
  
  I love this idea of collecting real usage data. There's a truism that the best software is developed by the people who use it, but that's never going to work for mass-market stuff like Word because the average user will never be a programmer. So this is what you might call "prosthetic empathy".
• Anonymous
  October 31, 2005
  Wow, how very 1984 ;)
• Anonymous
  October 31, 2005
  Save rarely used by end-users? I think I press Ctrl+S unconsciously every other word I typ in Word.
• Anonymous
  October 31, 2005
  Copy, Cut, Paste, Undo, Save
  
  Looking forward to see how far off we all are! :)
• Anonymous
  October 31, 2005
  The comment has been removed
• Anonymous
  October 31, 2005
  File >Print
  File> Save
  File> Save As
  File> Send to> Recipient as attachment
  Exit!
  
• Anonymous
  October 31, 2005

1. Paste
   2. Print
   3. Undo
   4. Copy
   5. Cut

• Anonymous
  November 01, 2005
  I disagree with everyone.
  
  My mother can't cut and copy and paste, and she's probably much more of a typical user than any of us. Most people will only save and print a document once in a session.
  
  My guesses: spellcheck, bold, underline, undo, print preview. But not necessarily in that order.
• Anonymous
  November 01, 2005
  The comment has been removed
• Anonymous
  November 01, 2005
  The comment has been removed
• Anonymous
  November 01, 2005
  I think backspace/delete is the most used command in Word - provided it counts as a command of course.
  
  After that perhaps underline is among the top 5 and I think spellcheck might be in that list too.
  
  Great blog by the way. It's fun to read about what you can do if you take usability and usability research really seriously.
• Anonymous
  November 01, 2005
  Wow, we've made it through 30 posts and no one's whined about privacy or misleading opt-in dialogs yet. Amazing...and refreshing.
  
  Now, the top five commands, in descending order:
  File Search
  Record Macro
  New Frames Page
  Document Map
  Outline View
  
  ;-)
• Anonymous
  November 01, 2005
  "MVPs who saw my talk at the summit and therefore know the answers, please refrain from showing off."
  
  By now I've completely forgotten. <smile>
  
  Nice blog. Very, very interesting reading.
• Anonymous
  November 01, 2005
  Hmm. Probably quite popular are Open and Save As. I say that because there are many people who don't know how to use Explorer so to put files on a floppy they open it and save it again on the disk.
• Anonymous
  November 02, 2005
  The comment has been removed
• Anonymous
  November 02, 2005
  1 ) Open
  2 ) Indent
  3 ) Header Style
  4 ) Save
  5 ) Undo
• Anonymous
  November 02, 2005
  Open
  Print
  Font
  Save
  Copy
  
• Anonymous
  November 02, 2005
  paste
  copy
  save
  print
  undo
• Anonymous
  November 02, 2005
  Open
  Save
  Save As
  Spell
  Close
• Anonymous
  November 02, 2005
  Undo
  Paste
  Copy
  Italic
  Save
• Anonymous
  November 02, 2005
  Close
  Exit
  Save
  Print
  Undo
• Anonymous
  November 02, 2005
  If I may be pessimistic:
  Centre
  Left
  Bold
  Italic
  Size
  
  Or if I should be optimistic:
  New Style
  Apply Style
  Update Style from Selection
  Save
  Hide/Show Paragraph Marks
• Anonymous
  November 02, 2005
  Exit
  Font Dropdown
  Font Size Dropdown
  Save
  Font Bold
• Anonymous
  November 02, 2005
  Word Count
  Check Spelling
  Find
  Print Toolbar Button
  Format/Font...
• Anonymous
  November 02, 2005
  Open
  Save
  Save As
  Print Preview
  Print
  
  And not sure if it counts as a command... but I imagine the recently used file list on the file menu would be quite popular, as well.
• Anonymous
  November 02, 2005
  Save
  Copy
  Paste
  Switch Document Windows (CTRL+F6)
  Find
• Anonymous
  November 02, 2005
  print preview - as I do this more than printing
  save - if setting the system to automatically do this counts
  word count - I know people who use this with frightening regularity
  copy/paste - they're really two halves of the same function to most people
  align > center - well i always do it a couple of times per document.
  
• Anonymous
  November 02, 2005
  Assuming that main-keyboard single-keystroke commands don't count (e.g., backspace, enter, tab)
  
  Selection (e.g., Selection.MoveRight, Selection.MoveLeft)
  Paste
  Save
  Open
  Exit
  
  
• Anonymous
  November 02, 2005
  Adding a guess, since there's almost 50 of them:
  
  font
  bold
  copy
  paste
  font size
• Anonymous
  November 02, 2005
  Given that Open, New and Print haven’t been deemed worthy of icon status in 12 they may struggle to make it to the top 5...First thing I will be doing is configuring the toolbar and bunging New and Open back to their ‘rightfull’ places.
  
  Let’s see: Undo, Paste, Cut, Font, Font size.
  
• Anonymous
  November 02, 2005
  I'd guess
  
  Buttons:
  Undo
  Save
  PrintPreview
  Print
  New Doc
  
  Keyb:
  Ctrl+z
  Ctrl+c
  Ctrl+v
  Ctrl+s
  Ctrl+Enter
  
• Anonymous
  November 02, 2005
  Ctrl-Z, Ctrl-B, Ctrl-U, Ctrl-V, Ctrl-X, Ctrl-S & their mouse&button & menu equivavalnts
  
  Do not forget, that for less-so-used commands your results are strongly biased by discoverability, the fact that there is workaround or alternative way to do it and a tradition (e.g. using tabs instead of moving left margin). The way you use your counted data you can establish strong positive feedback (in a "cybernetic" sense), so you take the commands-winners and make them more easily accesibly in Office12 - then no surprise, they will become even more popular in your next polls and on the other hand the commands-losers will get definitely worse results ever. After few cycles you burry any new, only-for-somebody features.
• Anonymous
  November 02, 2005
  The comment has been removed
• Anonymous
  November 02, 2005

1. New, which is done automatically for you when you open Word anyways
   2. Open, which people can call without Word even being open
   3. Save, every five minutes, just in case
   4. Change font, because everyone loves Times New Roman!
   5. Print, because it's hard to Xerox a laptop!

• Anonymous
  November 02, 2005
  The comment has been removed

• Anonymous
  November 02, 2005
  Ok, now for a few non-sarcastic guesses:
  Open Doc
  New Doc
  Bold Text
  Center Text
  Set Font

• Anonymous
  November 02, 2005
  Assuming that we're limiting it to just commands accessed through menus and toolbars (otherwise things like PageDown would be in the list) my guess would be as follows:
  
  Save
  Undo
  Paste
  Open
  Print
  

• Anonymous
  November 03, 2005
  Undo
  Print
  Paste
  Font size
  Spelling and Grammar

• Anonymous
  November 03, 2005
  I'm assuming that you're not asking about file menu commands, which is what I would expect most beginners to use, so I'm basing it on just the toolbar. So my guess as a standard user:
  
  - Zoom Page Width
  - View Final instead of (Final Showing Markup)
  - Font
  - Table
  - MS Office Word Help
  

• Anonymous
  November 03, 2005
  I have trouble getting beyond three:
  
  Bold
  Italic
  Underline
  
  I love a lot of the features of word ... style sheets are my friends, but in all the jobs I've had, I'm the ONLY person I have ever known who uses them. Everyone here formats using bold, italic, underline, and font size, so I should add that to my list.
  
  I don't think open and print will make it. You only open a document once, but you bold things in it a lot.
  
  I wonder if there are statistics which show what commands are used after creation of a new document as opposed to editing of an old document. Does "delete selection" count as a command?
  
  OK, so maybe I'll finalize my guess as
  
  Bold
  Italic
  Underline
  Font Color
  Font Size
  
  (Font color because many people I've seen don't understand revisions, and use font color to do revision tracking.)

• Anonymous
  November 03, 2005
  Open
  Close
  Font size
  Bold
  Undo

• Anonymous
  November 03, 2005
  The comment has been removed

• Anonymous
  November 03, 2005
  Can't believe no-one has guessed it!! So I'll try some others:
  
  Help/F1
  Select All/Ctrl+A
  Maximize
  Insert Picture
  the Window menu (many people are used to office 97)
  
  who knows!

• Anonymous
  November 03, 2005
  My guess on a top 5:
  
  Save
  Undo
  Print preview
  Bold
  Bullets

• Anonymous
  November 04, 2005
  Getting 5 in the right order is a bit harsh!
  
  Because most users seem not use Word in a sane fashion, I'll guess:
  
  Font size
  Bold
  Center
  Font [font face selection]
  Underline

• Anonymous
  November 04, 2005
  Bold
  Tab adjustments
  Font size
  Delete
  Paste

• Anonymous
  November 05, 2005
  I'd think:
  
  - Save ( for those pessimists )
  - Help ( people who have no idea what's going on )
  - Paste
  - Copy
  - Spell Check

• Anonymous
  November 05, 2005
  My guess:
  New
  Print
  Save
  Close
  Exit

• Anonymous
  November 06, 2005
  Most to least:
  Exit
  Print
  Undo
  Center
  Bold
  

• Anonymous
  November 06, 2005
  print
  save
  font size
  insert table
  bold

• Anonymous
  November 07, 2005
  Very interesting blog. Thanks for sharing your thoughts with us, Jensen.
  Here is my guess (saw your comment that nobody has guessed the right top five yet).
  undo
  save
  open
  font selection
  list item
  
  Not even close, am I?

• Anonymous
  November 07, 2005
  The comment has been removed

• Anonymous
  November 07, 2005
  This is the problem with posting at the same time each day: people start to EXPECT you to continue posting at that same time!
  
  So where's my Monday JensenH fix?? :)
  
  Very much looking forward to finding out the "real" top 5 most-used commands in Word.

• Anonymous
  February 06, 2006
  Another interesting series of&amp;nbsp;articles from Jensen Harris, sharing with us the rationale why Microsoft...

• Anonymous
  February 09, 2006
  Today, just thinking aloud...
  A minor design conundrum we face is as follows: based
  on the data we collect,...

• Anonymous
  February 19, 2006
  PingBack from http://jameslin.wordpress.com/2005/12/13/more-on-the-new-microsoft-office-ui/

• Anonymous
  March 27, 2006
  This is the seventh part in my weekly series of entries in which I outline some of the reasons we decided...

• Anonymous
  March 27, 2006
  
  This is the eighth part in my weekly series of entries in which I outline some
  of the reasons we...

• Anonymous
  April 05, 2006
  This is the sixth part in my eight-part series of entries in which I outline some of the reasons we decided...

• Anonymous
  April 07, 2006
  This is the seventh part in my eight-part series of entries in which I outline some of the reasons we...

• Anonymous
  May 23, 2006
  PingBack from http://kingsley2.com/archives/2006/05/24/office-2007-a-gentle-ribbin

• Anonymous
  May 24, 2006
  PingBack from http://pschmid.net/blog/1969/12/31/6

• Anonymous
  May 24, 2006
  PingBack from http://pschmid.net/blog/1969/12/31/7

• Anonymous
  May 24, 2006
  PingBack from http://pschmid.net/blog/1969/12/31/8

• Anonymous
  May 24, 2006
  PingBack from http://pschmid.net/blog/1999/11/30/9

• Anonymous
  November 02, 2007
  I really enjoyed having a beer with Andy Bounds last night. Andy is an inspirational communications coach

• Anonymous
  November 02, 2007
  I really enjoyed having a beer with Andy Bounds last night. Andy is an inspirational communications coach

• Anonymous
  May 07, 2008
  PingBack from http://dancingbunnies.wordpress.com/2008/05/07/create-alter-or-replace/

• Anonymous
  May 24, 2008
  PingBack from http://www.istartedsomething.com/20080525/hp-hdx-dragon-notebook-giveaway-starts/

• Anonymous
  May 26, 2008
  PingBack from http://www.wowgoldip.com/2008/05/27/hp-hdx-%e2%80%9cdragon%e2%80%9d-notebook-giveaway-starts-today/

• Anonymous
  January 03, 2009
  PingBack from http://www.tdaxp.com/archive/2009/01/02/the-science-and-art-of-user-experience-research.html

• Anonymous
  May 29, 2009
  PingBack from http://paidsurveyshub.info/story.php?title=jensen-harris-an-office-user-interface-blog-inside-deep-thought-why

• Anonymous
  May 31, 2009
  PingBack from http://woodtvstand.info/story.php?id=4825

• Anonymous
  June 08, 2009
  PingBack from http://quickdietsite.info/story.php?id=4055

• Anonymous
  June 15, 2009
  PingBack from http://unemploymentofficeresource.info/story.php?id=1249

• Anonymous
  June 18, 2009
  PingBack from http://thebasketballhoop.info/story.php?id=2698