通过

Let's experiment!

  • 项目

We're excited to announce a new initiative here on the GPDE team -- Experimentation. We're going to be shipping a key part of Microsoft's new Experimentation Platform, which is a system for testing out different web designs and other experiments on real live users. It will initially be used by MSN and other online sites from Microsoft, and we have plans to eventually roll the service out to any developer using Windows Live.

Experimentation is both a technology and a philosophy. The technology is based on randomized experimental design, in which a small set of users get a new version ("treatment") of a site or service, while the rest get the old version ("control"). The system measures how effective the experiment is (by evaluating metrics designed by the site owner, like number of clicks on a page or number of purchases), and then does some analysis to see if the results are statistically significant. The platform also includes the ability to quickly auto-abort an obviously failing experiment, and even suggest other hypotheses to be tested by new experiments.

The philosophy of experimentation is simple -- "let the users decide". Traditionally, decisions are made based on the HiPPO (the Highest Paid Person's Opinion). The Experimentation Platform means our customers can instead make decisions based on the results of controlled experiments. The hippo stress toy (see picture) is for those moments when an opinion-based decision happens anyway.

Click over to exp-platform.com to learn more about experimentation at Microsoft and read some recent papers. And check out the open jobs on our recruiting site.

Comments