Followup to post re: community preview philosphy

Thanks for the comments/feedback I received.  A few additional comments (I'll try to keep this one short, or at least shorter).

Josh Ledgard  questioned the need for every/daily builds.  I use the word “every” to make a couple points, one internally and one externally. 

The internal point is that “shipping” these builds - with all necessary extra info like content and quality - has to be automatic/free.  If it’s not, i.e. we have to decide which build, or do extra work to “polish” the build before it ships, there will be resistance to doing them - the daily build is pretty ingrained into how we do things and keeping forks (and dev machines set up) for previous builds adds lots of work.  We also get into gnarly conversations about whether to fix some hideous bug in one area before we ship it - those are gnarly conversations because up until we get close to shipping, there's almost always at least one hideous, embarrassing bug in every build - the choice is really to lockdown the build while we fix the hideous bugs, or fix it in the next build and risk that the new crop of hideous bugs is worse.  The longer we stay locked down, the longer most devs go without having their work subjected to the next testing gauntlet – they can’t check in, can’t build against all the other changes, etc..  Someone commented that Whidbey is complex – that is absolutely reflected here at the daily, product development level.

Externally, I want to make sure that we set expectations that the build from today is not necessarily better than the build from yesterday and that the casual observer should think twice before installing.  Our builds do not have a consistent incremental increase in quality.  There are spikes and valleys.  There are valleys even late in the release cycle, e.g. we send a beta out, get feedback, check in improvements across the product, and break things (which we then fix).  I want customers to see lots of builds coming out so they understand the risk, vs. seeing fewer builds and thinking (erroneously) that each build is good and “guaranteed” to be better than the last one. 

All that said, I’m more committed to frequent, automatic, and public, than “every build”.  I just think aiming for every build is the best way to get there – otherwise we’ll think incrementally.  It's easy to scale back, e.g. from every day to once a week, or to whatever we end up seeing customers do. 

I do not mean to imply that there is zero order within our teams.  Over time the builds absolutely get better, we have a set of build verification suites that are run for every build,  we have tools in place to help make “breaking changes” easier to manage across the division, and there are always people looking at process/efficiency improvements.  In many ways, what our teams do is amazing.  But the complexity is very high, the % of tests we can run against each build is relatively small, and bad bugs get through.

The feedback about “must always pass”  tests is good.  That is what our build verification tests are meant to show, but reading this prompted me to think about three things: 1) maybe we’re not looking at the right things in the bvt’s.  2) maybe we need a more comprehensive set that takes longer to run but which still completes within a day or so.  3) can we get community help here?  i.e. perhaps we augment what we’re doing with a suite that the community provides.  That would be a great rallying data point for the division – do our daily builds at least reach the bar for what customers are willing to install?  I’m not looking to pass work off on others, just thinking about where I can blur the line between internal and external so that everyone ends up better off in the end.  I don’t remember if I’ve written about this before, but I’m also interested in the possibility of developing many test cases in public workspaces so people can at least see them.  Maybe bvts are the pilot we’ve been seeking.

Thanks for the comments re: encouraging people to try this.  That’s exactly where we are (although we’re also lucky to have strong support from Eric Rudder on down to help the division get started and survive through the inevitable glitches).  People are pretty excited about being more aligned with customers’ needs and wants.   I haven’t met a person who thinks this is a bad thing.  The concerns I have heard are very legitimate:  build quality is variable, how do we do this in a way that doesn’t cause customers to waste their time or  become dissatisfied, it’s extra work to gather/publish some of this information.  Those are all things to work through, vs. reasons for not doing it.

Comments

• Anonymous
  May 13, 2004
  I think the only obstacle to "every" build from the consumer perspective is size. If you can deliver an incremental "every" build in, say, under 50Mb, I would say go ahead and post every build. However, if the builds are going to be 2Gb+, then there's just no way many people would go through the trouble of downloading, burning, and installing the monstrosity on a daily basis. (heck, even on a weekly basis!)

• Anonymous
  May 13, 2004
  We're not trying to achieve a scenario where any one customer installs on a daily basis. It's more about publishing them daily/weekly, and enabling customers to install when it makes sense to them. My hope - and I agree that size will impact this very significantly -is that most builds get enough installs to generate a customer-based assessment of "was this a good build". That way another customer who wants to install a recent build can look back and pick the best one based on the information available.
  
  I said this in the other post, but the scenario for me is a customer reports a bad/blocking bug to us, we fix it, and the customer can install the fixed build within a few days.

• Anonymous
  May 13, 2004
  The comment has been removed

• Anonymous
  May 14, 2004
  I develop with both C# and VB.NET, but I like VB.NET
  to be better because I am old VB developer.
  I just installed VS.NET 2005 (Technology Preview), and
  this my prompt feedback
  
  -----------------------------------------------------
  I surprised that I found productive feature in C# and
  not in VB.NET, this features:
  
  1) Refactoring
  I tried Refactoring in C#, It is really very
  productive
  feature.
  
  2) IntelliSense/Auto Completion (Keywords)
  
  Now only in C# debugger feel the Language keywords
  like
  (private, public, foreach ...), and auto complete it
  to
  you.
  
  However VB.NET have longer Keywords such as
  (MustInherit,
  NotInheritable,.)
  
  -----------------------------------------------------
  I dreamt that I can find these features in VS.NET, but
  
  unfortunately I frustrated when I found this feature
  in C# not VB.NET :(
  
  Microsoft always say that it concentrate on
  productivity features in VB.NET, How does VB.NET 2005
  lack these very productive features ?!
  
  Looking forward for your reply.
  
  Amr Essam
  www.Verizon.com
  Consultant & Team Lead
  MCSDT + MCT
  Dallas, Texas

• Anonymous
  May 15, 2004
  Could it be cause VB.NET team focused on E&C, which is missing from C# as C# team focused on the refactoring and other little nice things.
  
  VB.NET has long keywords probably cause that's how VB seems more approachable as it resembles something you know (english).
  
  What I don't understand is that why there is no single "codebase" for VB.NET & C#, and then the language syntax differences between them would be just glued on using inheritance or whatever. Then when E&C was introduced into the "root code", both VB.NET and C# would have "inherited" it. I am probably over simplifying things in my mind..

• Anonymous
  May 19, 2004
  The comment has been removed

• Anonymous
  May 24, 2004
  As a keen .Net developer without an MSDN Universal Subscription ts very frustrating not being able to get my hands on the VS2005 releases - are they going to make it out for general download?

• Anonymous
  June 23, 2004
  Just to reiterate what Martin Dolphin said - it's great that you're working towards making more of your builds available, but it would be even better if something were available to a larger portion of the general public. Not necessarily these daily builds, but something. Last I heard, the public beta was scheduled for "June" - June's nearly over and there's no new word, that I've heard. Any news on that?
  
  It's very frustrating - I've been positively drooling for C# with generics, and I can't get at it anywhere! Preview access to VS2005 excludes me, and the Mono windows installer doesn't include their generic-enabled preview compiler (I'm not a sufficient guru to build Mono from source, especially on Windows).
  
  

• Anonymous
  January 24, 2008
  Also advance cash loan payday wired advance cash day loan pay

• Anonymous
  February 02, 2008
  In need of advance cash day loan pay cash advance service

• Anonymous
  May 29, 2009
  PingBack from http://paidsurveyshub.info/story.php?title=mark-cliggett-s-weblog-followup-to-post-re-community-preview-philosphy