Learning as an engineer -- rising from the ashes of failed projects
Recently a senior program manager asked me for some reading suggestions and I wanted to share a book that had a big impact on how I view projects that I have worked on. The book is called "To Engineer is Human: The Role of Failure in Successful Design" by Henry Petroski (I have no connection to the author and never met him). This is one of my favorite books to give to engineers for holiday presents and I think I’ve probably given a copy to most of the senior managers on the Office team.
The book is a wonderful history of some engineering projects. The primary thesis that is well supported by the evidence is that for every successful engineering project there is a series of failures that preceded it. Mr. Petroski is a wonderful writer and the book is an easy and fun read. Of course one attraction is that the book talks about the failure of the Tacoma Narrows bridge, which is right by Seattle and an amazing site both in person and when you look at the famous pictures of its collapse. The book details other even more tragic events such as the Kansas City Hyatt walkway collapse. It shows how small design mistakes can be overlooked and also how future designs benefit from the failures. Often people who are not engineers really don’t quite understand how you learn and improve as an engineer by making mistakes, and obviously the challenge is that if your failures cause damage it is really unfortunate.
I think a lot about this relative to the security issues we face in computer software today—we did not design for these and while we are doing everything possible to update old software the real benefit is coming from the new generations of software that were designed from the beginning to work in this environment (for example, in the 2 years that Office 2003 has been out we have only had 2 critical updates). I don’t want to make any excuses at all so don’t misinterpret that, but the learning that has taken place in our industry over the past 5 years has been immense and necessary for us to build products like Office 2003.
When I think about software projects that I have worked on from an OS in CS 415 at Cornell through an object database in graduate school all through Microsoft projects, the notion that failure often precedes success is one that is near and dear to me. Our industry is of course filled with failures that went on to become the basis upon which much learning took place (like the Newton to the Palm Pilot, for example, or the new TabletPC learning from the old PenWindows).
The reason this book resonates with me is because my first project at Microsoft was a pretty colossal failure. I was hired into a very small group (in business school lingo, the group would have been called a “tiger team” – a hand-picked group of experts with a specific short term mission). I don’t know how I ended up in this group since it was filled with incredibly senior developers and I was really intimidated. In any event, the mission we were given just before I started at MS was “build an application framework for Windows”. It sure seemed easy enough—we had a lot of great folks and even little me had just worked on Smalltalk frameworks for 2 years so we collectively thought “no problem”.
We worked for about a year and had a rather complete framework. We were pretty confident in our ability to use it to build robust applications in short order. So we decided to spend a month (a whole month!) building applications. Microsoft Money was under development so I figured I could spend my month building Money, which seemed simple enough. The framework had graphics, persistent objects, a UI model-view-controller framework, etc. ¡No problema!
After a month I had a fantastic application – it was able to successful draw a check on the screen. Needless to say our framework was too big, too slow, too complex for anyone to really use. It was a dismal failure despite all the right ingredients and doing all the “right” things in object oriented design. In fact we had overdosed on objects and had become full-blown oopaholics. Rico has a blog and video on the abuse of object-oriented design worth checking out (Rico was working on the compiler at the time I was working on the class library)
As a result we had a rude awakening. Our manager’s manager’s manager was getting frustrated with progress and we were under the gun. We had a big meeting to review our lack of progress and had to admit that we had not made 12 months of progress over the last year. We had to regroup.
I received my first Microsoft lesson: just because you messed up does not mean your career is over. What matters is how you dust off, what you learn, and if you can put that into practice. Microsoft holds no grudges. The boss said “ok you made a mistake, let’s get moving and don’t make that same mistake.”
We regrouped. We spent a couple of days talking about what we learned and very quickly came up with a number of “rules” about how we would really design a great class library for Windows. I can’t quite remember them all but some of them were:
- We’re building a class library, not a compiler test suite, so there is no need to try to use all the features of C++.
- Don’t build new abstractions on top of the existing abstractions in Windows. Folks will need to know Windows to use the framework anyway.
- Developers will want to call Windows so don’t cache any state in the classes that is duplicating state maintained by Windows (for example, parent child window lists, styles, etc.)
- Performance matters from the beginning so don’t go and use fancy stuff that has a fixed overhead that you don’t know if you need (like using virtual functions everywhere for example).
And so on. Once we had these lessons in place we actually developed the whole framework in a few months and were able to ship it with Microsoft C++ 7.0 (our first C++ product). We designed in parallel the next release along with all the graphical tools that became Visual C++, which itself became Visual Studio over time. The class library, Microsoft Foundation Classes or MFC, became a great tool used by tons of Windows developers around the world. I think my proudest moment was walking up to a really tall guy at a DEC booth in a partner section from the University of Ilinois at a tradeshow demonstrating an alpha of a Windows network application called a “browser”, I think his name was Mark, and asked him what tools he used and he said “MFC of course”. Yes!
When I think back about the dismal failure of our first library and how as a team we seemed to have developed a massive case of second-system syndrome for a first system (a mean accomplishment) and how we regrouped, learned lessons, and put those into play, I realized that without that first failure we never would have developed the success criteria that allowed us to build MFC.
The fact that Microsoft offered an environment where we could practice the lessons detailed by Henry Petroski made the whole experience even better.
And most of all I am thankful that I got to be part of an amazing team who mentored and pulled me through these experiences as a new hire. I really want to thank all the people on this project that taught me so much. Thank you for letting me be part of our collective failure and success!
--Steven
PS: I used links to Amazon, but by all means buy your books from your favorite bookseller.
Comments
Anonymous
August 19, 2005
TrackbackAnonymous
August 19, 2005
Steven Sinofsky, VP
of all things Office, posted a great blog
entry last night on the important...Anonymous
August 19, 2005
That was a very nice entry. I hope you post more anecdotes like this one and recommend more books in the future. :-)Anonymous
August 19, 2005
Thanks dan!Anonymous
August 21, 2005
> Folks will need to know Windows to use the
> framework anyway.
Yes and no. First, perhaps you will be glad to know that folks didn't have to know Windows in order to use MFC, up to a point. After a few decades on mainframes, minis, Unix, embedded systems, etc., some reasons arose to do some Windows programming. MFC turned out to be convenient. (Later I learned that VB is even more convenient and it wasn't Basic any more, but that's irrelevant now because VB.Net isn't VB any more either.) So anyway, up to a point, it was possible to do Windows programming using MFC without learning Windows APIs and the WinMain() message loop and stuff like that.
But then I grew too comfortable with MSDN pages documenting MFC classes. Where some parameters were DWORDs or ints, and MSDN didn't specify any limits, I assumed the values could be 32 bits. But under the covers, without documentation or warning, MFC was forwarding those values to Windows APIs that had limits of 16-bit or 15-bit values. Your colleague Raymond Chen said this should have been obvious to me and I should have known not to (for example) compute a window height of 40000 pixels and try to let the mouse control the scroll bar through that entire height. Well I didn't know, MFC didn't say and MSDN didn't say.Anonymous
August 21, 2005
I definitely agree with you.
When we were doing MFC we just didn't want to create new abstractions. So if you weren't quite sure how to do graphics programming (stuff like manipulating bitmaps, pens, etc.) then you could count on reading Windows documentation and we would not have to duplicate everything out there for Windows--which would have been impossible since books, courses, etc. were growing so very fast. But for the most part, you could use AppWizard, ClassWizard and stick to straight MFC programming for a ton of work--I know that Mark and team did that for their first browser :-)
I don't know if it should have been obvious, but early on we did make a bet on using 32 bits because the first version of the MFC GUI classes worked on Win16 and Win32. Historically, it was actually an eye-opener for the Windows team since we went and "widened" WINDOWS.H before they did and found a ton of places where the new NT-based APIs were just different than the Win 3.x APIs--all of that got fixed before the very first NT Developers conference.
Perhaps the most classic example of this was the CEdit control which was limited to something impossible to determine around 30K. We used widened values which often confused people. We did make a note in the API reference but judging by the number of questions, it was not a big enough note.
Thanks so much for the comment,
StevenAnonymous
August 22, 2005
Steven,
This is a really nice post. I especially liked the 4 rules that you were able to remember. Here's hoping you remember a few more :)
- ArunAnonymous
August 23, 2005
Thanks! I'll keep trying to come up with other ones!Anonymous
August 24, 2005
… for Architects
Nick Malik – Enterprise Architecture Agility
Roy Osherove – [Audio Interview] Ingo...Anonymous
August 30, 2005
I encourage you to read "Learning as an engineer -- rising from the ashes of failed projects" from Steven...Anonymous
August 30, 2005
I encourage you to read "Learning as an engineer -- rising from the ashes of failed projects" from Steven...Anonymous
April 13, 2006
The comment has been removedAnonymous
September 09, 2007
elementary science fair experimentsAnonymous
September 25, 2007
free myspace animated backgrounds codesAnonymous
February 28, 2008
PingBack from http://msdnrss.thecoderblogs.com/2008/02/28/interesting-people-in-seattle-part-1-steve-and-javed-microsoft-legacies-and-oba-mashups/Anonymous
August 15, 2008
PingBack from http://informationsfunnywallpaper.cn/?p=812Anonymous
August 15, 2008
PingBack from http://www.easycoded.com/huh-e7-windows-7Anonymous
August 15, 2008
PingBack from http://housesfunnywallpaper.cn/?p=960Anonymous
August 15, 2008
PingBack from http://hubsfunnywallpaper.cn/?p=874Anonymous
August 15, 2008
PingBack from http://blog.a-foton.ru/2008/08/huh-e7-windows-7/Anonymous
June 15, 2009
PingBack from http://debtsolutionsnow.info/story.php?id=8223Anonymous
June 18, 2009
PingBack from http://patiocushionsource.info/story.php?id=2358