Whidbey betas

Robert Scoble's blog asked the “how do I get the Whidbey beta?” question.  I can't answer that yet, but I can tell you about the direction we're headed in for betas in the Tools division.  There are 3 areas that we want to improve:

- broad availability:  in the past we've done limited betas and it has taken a combination of luck and magic to participate (hence the How Do I Get It? question).  We want to make this a lot more clear and make it simple for people to participate.  This also probably means larger #'s - if we have simple mechanisms I'd expect more people to do it. 

- more frequent updates:  Out of the best of intentions - providing quality builds - we've gotten to a point where we extrude new builds very infrequently.  We don't think everyone needs frequent updates, but we know that there are people who would use them, for example to verify that a bug has been fixed or to get unblocked if they hit a showstopper.  We are going to experiment with providing more frequent updates.  Some of these will not be of the same quality as the official/big beta drops we have done in the past, simply because running full test passes takes weeks in some cases.  People may see significant regressions because the chosen build had a bad bug.  There's a fair amount of tension about this idea internally - people worry about the regressions, the possibility that people will get burned installing builds that don't work as well as the last one, and whether anyone will use these anyway.  My view is that people don't have to install the updates, that we can provide some guidance on what does/doesn't work (perhaps in conjunction with a community-based feedback mechanism, e.g. “I installed this build and it's great/sucks”) and that in the long-run both the development teams and the external customers will find these of value.  If anyone has thoughts on the quality/frequency tradeoff, I'd love to hear them. 

- beta programs that are inline with our retail packaging plans.  A couple examples 1) it might be reasonable for a VS Pro user to have to be part of MSDN.  2) but, if someone spends $90 on VB Standard, and they want to participate in the next release, they shouldn't have to have an MSDN universal subscription.  This gets into packaging, which I'm not an expert in and which typically changes multiple times over the course of product development.  But the basic gist is that if you have shipped version today, there ought to be a beta process tomorrow that is consistent with your needs/expectations. 

As I said, there's some tension around all of this - e.g. quality vs. frequency above.  There's also tension about scaling the numbers up - people want to be responsive to bug reports, newsgroup posts, etc., and the concern is that we could get overloaded, not be responsive, and not make great progress shipping either.  Again, my own view is that it will work out ok as long as we're open about what we are doing and about our challenges as we go.  We'll see - in the coming months we will take concrete steps on all of the above. 

And pretty soon I hope to have a crisp answer on “How do I get the Whidbey beta?”

Comments

• Anonymous
  January 15, 2004
  The comment has been removed
• Anonymous
  January 15, 2004
  Drop frequency ranged from once a day (near the end or to fix a particularly onerous bug) to a few weeks.
  
• Anonymous
  January 15, 2004
  Yes, that's the spirit of what we want to do. I think once you have the right mechanisms in place it, the process is self-organizing. But it's a shift from what we've done and it's easy for people to say "But..."
  
  Thanks for the pointer - I'll definitely check it out.
• Anonymous
  January 15, 2004
  Beta participation is very important for product lifecycle and I'm glad to hear that more frequent builds will be released.
  
  It would be nice to see a Wish list/bug track integrated into betaplace as well.
  
  You guys do great work, and its nice to be able to leverage your work sooner rather than later
• Anonymous
  January 15, 2004
  It's pretty easy to find on peer-to-peer and on the normal IRC beta groups.
• Anonymous
  January 16, 2004
  Mark, how does your drop proccess change for the (i assume) less frequent drops of the whitehorse/burton (SOA) capabs/functionality? My focus is to use the same in Wall Street WAN/LAN Asynch-Messaging environments...
• Anonymous
  January 30, 2004
  Sorry, I missed the question on whitehorse/burton earlier. What are whitehorse and burton? :-) I'll answer this question more generally. Over time we want every customer to have good visibility into our development process, regardless of what features/sku/product they are using. It may be appropriate to do drops more or less frequently depending upon the customer segment , but the general view here is that we should make things available more often - that puts customers in control of how often they pick them up.
• Anonymous
  April 03, 2004
  As well be hanged for a sheep as for a lamb.