Microsoft has been quietly building a platform to help its own product teams — and ultimately, those from other companies — turn product planning more of a science and less of a black art.

Microsoft calls the test bed the Microsoft Experimentation Platform (ExP). Here is how the EXP team describes its mission on its Web site:

“The Experimentation Platform enables product groups at Microsoft and later on will enable developers using Windows Live to innovate using controlled experiments with live users. The platform enables testing new ideas quickly using the best-known scientific method for establishing causality between a feature and its effects: randomized experimental design. The basic methodology in controlled experiments is to expose a percentage of users to a new treatment, measure the effect on metrics of interest, and run statistical tests to determine whether the differences are statistically significant, thus establishing causality.”

The chief experimenter behind this initiative is General Manager Ronny Kohavi. Kohavi joined Microsoft in 2005 from Amazon.com. At Amazon, he was the director of data mining and personalization. He joined Microsoft’s Natural and Interactive Services Division (NISD) to build a system that would map user activities to intent using machine-learning algorithms.

“In the first few months, I realized that few tools existed at Microsoft to run live experiments and make data-driven decisions based on user actions,” Kohavi told me, via e-mail. “I was being asked to build something with limited ability to iterate quickly and test ideas with controlled experiments, so my initial reaction was to build a small system to run controlled experiments within my project.”

Full Story At Source

Send via e-mail | Submit to Digg | Add to Live Favorites