Living complexity
Brad's Carl Sagan-style post on code size stirs up some familiar questions about complexity management here at MacBU. With every release of Office, we remove, rewrite and add new code. Often we can remove significant amounts of code without losing functionality – say, remove use of a custom HTML renderer when the OS offers a built-in one instead. I recall how many of us took it as a badge of pride during the Carbon transition of Office v.X that after a year of intensive coding our net contribution in terms of new lines of code was significantly negative. Those were the days, hacking and rewriting. But of course, as the total functionality of the suite increases over time, the total volume of code used to build it tends to inevitably follow. There has been a lot of functionality added in twenty years.
Is it necessary? It's pretty easy to find people who agree that Office has more functionality than they personally need. Where the agreement tends to fall apart is when you ask what the first feature to be cut should be, at which point one person's "bloat" is another's "only feature I use every day". It's also nice to know the functionality and compatibility is there, as you find the need or the advantages of it. I use all of our apps every day, not to test but just to get work done – Exchange-based mail and calendaring in Entourage, analyzing data and lists in Excel, IMing over corp chat – it's an interesting job being able to use the software every day to talk about the job of making the software. My Office skills were pretty minimal when I got here, but (and some coworkers might smirk at this) they've come a long way. You may not have heard of (say) pivot tables, but once you start using them for certain kinds of analysis you can't imagine how you ever lived without them.
Of course, our goal isn't to just pile features on top of features, but to harness the increasing power in the suite with an effective and focused user experience that puts functionality in reach and in context. In some cases that can be pretty easy, but generally it's actually pretty hard, and something we have to think about with everything we do.
Is it slow? Generally, there's no linear relationship, where adding 10% code means everything is 10% slower. There's quite a lot of engineering happening down at the metal to swap in the currently executing code as the application runs. The problem is in how much work the application is trying to do – if it's trying to do a lot, it's necessarily going to take longer to do. When we work on performance bottleneck areas (such as boot speed) that is generally what we are looking at, i.e. do we really need to be doing all this now? For this release, running native on Intel is orders of magnitude more important for speed than trying to trim raw code size, for example.
Is it sustainable? It's important to understand that while the code has been written and expanded over the past twenty plus years, it has also been tested and validated for twenty plus years. Insane amounts of testing. Stabilized or "baked" code provides a baseline for correctness and quality – as new changes introduce unintended problems (regressions), we know we just need to hunt down which new change is the culprit and understand how it needs to be redone. The real impact is on productivity: while correct, that baked code can vary wildly in how well factored it is (well factored being the opposite of "spaghetti"), and therefore how sensitive to regression. We definitely have some pasta mixed in as part of our overall diet, and we have to develop strategies for dealing with that. Sometimes it's best to simply say "do not disturb the water", but that's not always an option (say, for example, you decide to blow out the Excel cell table.)
Shouldn't we just rewrite it all from scratch, and "do it right" so that *all* of the code is perfectly well factored according to modern sensibility? JoelOnSoftware made some great observations on that topic in this article: Things You Should Never Do, Part I. (I believe link-master Brian sent that to me originally. Pyramid was before my time here so I'll leave it to others to comment on whether Joel's observations on that project are fair and accurate or not.)
Some people keep souvenir copies of tech editorials from ten years ago predicting that Microsoft will be unable to release any further versions of Excel, because the code base has simply gotten too big and Microsoft will have to rewrite it from scratch. That was quite a few releases of Excel ago.
Joel points out that code doesn't rust, but the complexity problem is real because massive change is inevitable if you are trying to keep your software on the leading edge – especially on the Mac. We plan to continue shipping Mac software into the foreseeable future, so it is easy for us to make major architectural decisions with the very long term in mind. Where it gets hard is balancing this against short term customer need – what resources can we apply to deep rearchitecture or local refactoring, given that these resources will not be available to solve the most pressing problems that are actually visible to customers? While we made significant investments in future capability earlier in the current product cycle, right now that customer need is pretty extreme and we're 100% focused on getting this release out into the market.
Finding this sort of balance overall, and dealing with this level of complexity in general, is clearly our defining challenge with Office. It takes some time to get your head wrapped around it all. We might go days without seeing a particular new developer, and we'll have to send a veteran search and rescue squad into their area of the code to pull them out. Of course, it doesn't just impact developers, it's in the density of the test cases, the maintenance of the automation, the range of functionality and interactions our designers have to understand, the scope of documentation, the pace of change. Managing complexity is where we live. It's not for everyone, but fortunately I find it all pretty fun at the end of the day... gives my head something to do.
Comments
Anonymous
November 10, 2006
waaaah! waaah! waaah! If you didn't want to write code than do something else but don't ask everyone to feel sorry for you and to ignore the huge gaps in between Office releases.Anonymous
November 10, 2006
I'm just getting involved in some testing projects where I work, and right now much of what we do is manual testing of use cases by having a human run each scenario. I think I've read that much of the testing for Windows code is automated -- is the same true of the Mac Office code? Is it automated at all levels (unit, integration, systems and client acceptance, and perfromance), or just at the lowest levels? I'm assuming that if you do automated testing that you use an internal suite of test tools -- do you know of any good ones for Mac software? Just curious - thanks!Anonymous
November 10, 2006
Thanks for this post - it answers some questions I'd posted in the previous thread.Anonymous
November 10, 2006
Awesome post Geoff... Great job! Also, related to the subject of Microsoft Office you might enjoy reading my blog post of November 7th.Anonymous
November 11, 2006
Anthony... hmmmm... did you actually read the blog? Because you appear to have totally missed the point. He is not complaining about having to code; he obviously loves coding. The challenge is coding efficiently, and writing lean applications that are feature rich. To add functionality while REDUCING code is a huge success for any programmer, and addressing massive amounts of unnecessary code -- which makes applications larger and run far slower -- is the point of this blog entry. Hope this helps.Anonymous
November 11, 2006
> It's pretty easy to find people who agree that Office has more functionality than they personally need. Where the agreement tends to fall apart is when you ask what the first feature to be cut should be, at which point one person's "bloat" is another's "only feature I use every day". It's also nice to know the functionality and compatibility is there, as you find the need or the advantages of it. Absolutely. The number of people who complain about the "bloat" of professional apps and then fail to understand that there's more to the difference between Word and TextEdit than size alone! If people are upset about perceived bloat and the slow release cycles of major apps like the MS Office suite and Adobe's Creative Suite too, I seriously advise them to look elsewhere for their needs. If you're doing something simple, a simple app is absolutely the right way to go for many reasons. But don't assume all of us are in the same boat, it's not all writing family letters in Word and doing basic charts in Excel. I myself am really looking forward to what's new in Office 12, as I am in all the big apps I use on a regular basis. A lot of it has saved my day over the years and the fact this continues and on my favourite platform too is very reassuring! Sometimes Office's development seems to resemble a new edition of the Oxford Dictionary. But many of us need our top tier apps just like some need the full dictionary.Anonymous
November 11, 2006
Why not componentize Office and it's applications?Anonymous
November 11, 2006
A good fix for many of these problems is to use a plugin or modular approach, where users load or turn on/off modules that they need. Faster, leaner, without any feature compromise...Anonymous
November 11, 2006
We already do this to some extent with mini-apps like Equation Editor and Org Chart. One potential issue with plug-ins is that many users have a difficult time figuring out that they need additional plug-ins to do something. I've lost count of how many times I've run into a user who wants to use the functionality in those mini-apps, but can't figure out how to do it because they didn't install it originally. I would think that there are other potential issues (testing comes to mind immediately), but user experience is my particular speciality.Anonymous
November 14, 2006
You say: "Often we can remove significant amounts of code without losing functionality – say, remove use of a custom HTML renderer when the OS offers a built-in one instead." I say: "then how about removing the use of a custom spellchecker when the OS offers a built-in one instead?"Anonymous
November 14, 2006
WHEN WILL THE OFFICE 2007 PLUGIN FOR OPENEING OFFICE 2007 DOCUMENTS BE AWAILABLE FOR OFFICE MAC?Anonymous
November 21, 2006
Thanks for all the comments. Kirabug - the degree to which testing is automated varies quite a bit by code area; we're pretty focused on increasing it at all levels. We do have a separate team dedicated to building such tools internally. I've asked real testers to comment further (though they're swamped...) Sheamus - thanks, enjoyed your blog. Nathaniel - we have certainly looked at that. If I can ask, are you interested for personal productivity reasons (i.e. consolidate your personal dictionaries) or for reasons of platform/experience consistency? Dr. T - We’re still in the thick of our new file format work; unfortunately I don’t have a new date to share on that at this time.Anonymous
November 24, 2006
Well that's gerat - did you know that people at microsoft are working with office 2007 beta? And do you also know, that they force me to work with thier files, but can't because of an incompatibel document format? no - well thank you! i'm looking forward to the mac expo in jannuary, to see some of you guys, that are responsible for these kind of problems!Anonymous
November 30, 2006
Appreciate the hard work you guys do. Keep up the good work. (p.s. Using the internal dictionary, would save you time by allowing you to leverage an existing canned item.)