Telemetry Myopia: The Toyota “Fail Safe” system Debacle and Google cavalier attitude toward Privacy
For the past few weeks members of my team and I have been meeting with key individuals in Bing, Microsoft Online services, and Windows. In those meetings we have focused on rich telemetry. We’ve talked about technologies to collect data, privacy concerns and how to architect to support user privacy, and the data platform needed to process terabytes of data per day.
One of the most well known of our telemetry systems we call Watson but is known to end users and OEMs and Windows Error Reporting. This telemetry system sends us post crash debug information when something fails in Windows. It is a key source of data for us to identify compatibility issues with applications and drivers as well as needed bug fixes in Windows and all our applications. In Bing we have all the ad click data and other rich logging on par with Google. That data is vital to running and improving Bing in this hyper-competitive market of search however we also feel strongly about end user privacy and have put in place what we feel are much stronger privacy policies than our competition. On this point just ask Asa Dotzler, Mozilla’s Director of community development about his reaction to a recorded statement made by Google CEO Eric Schmidt, ""If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place... If you really need that kind of privacy, the reality is that search engines, including Google, do retain this information for some time." See the video here and read a good article here, “ Mozilla exec suggests Firefox users move to Bing, cites Google privacy stance. Deal with Google provides most of Mozilla's revenues”
The point of this blog however is not to explain completely about the importance of telemetry in services or to dig into privacy policies. I saw an article a few weeks back that made me think about the risks of over-reliance on telemetry. In February ABC news ran a story about the results of some tests by David Gilbert, an automotive technology professor at Southern Illinois University. In this story, the professor showed not only how could the Toyota cars experience sudden acceleration but that the problem might not be logged into the car’s black box system and fail to trigger safety mechanisms.
Gilbert says the flaw "prevents the car's onboard computer from detecting and stopping certain short circuits that can trigger sudden speed surges." ABC News adds, "he can recreate a short circuit in the electronic throttles of Toyotas that can create a surge of acceleration but can't be detected by Toyota's electronic sensors."
ABC News: Expert says electronic design flaw to blame in runaway Toyota models
February 22, 2010
In another article by the Associated Press that commented on Toyota’s secretive nature about their Black Box data the authors wrote.
SOUTHLAKE, Texas - Toyota has for years blocked access to data stored in devices similar to airline “black boxes" that could explain crashes blamed on sudden unintended acceleration, according to an Associated Press review of lawsuits nationwide and interviews with auto crash experts.
The AP investigation found that Toyota has been inconsistent - and sometimes even contradictory - in revealing exactly what the devices record and don't record, including critical data about whether the brake or accelerator pedals were depressed at the time of a crash
Toyota secretive about 'black box' data
Curt Anderson and Danny Robbins Associated Press Writers; March 7, 2010
And to be fair, Toyota has come out with a statement rebutting the findings of Professor Gilbert that you can read here. In short, they say his re-wiring to create an environment with sudden acceleration that wasn’t logged in their systems was not one that could occur through regular use of a car. In other words his results were flawed.
My take away from the Toyota mess and the “Fail Safe” system is to be careful not to completely rely upon telemetry. Don’t get me wrong, I see rich telemetry data as changing the way we develop, test and ship both software and services. It allows us to be leaner in our processes and further enhances our ability to test in production. The thing is, it is just a data stream.
When you take your eye off the customer, when you rely too much on data and don’t hear the complaints or concerns of end users you can end up making foolish statements about privacy such as the Google CEO or you can become secretive with your data and insular like Toyota. The measure of the success of a product is always in the hands of the customer making the purchase. Telemetry is great as long as it does not overcome our other senses making us myopic.
That’s it. I just needed to document this thought and the links. When I’m done with my research on Microsoft telemetry systems and we design what we are planning for Office 15, I’ll try to post again on this topic with design recommendations and personal lessons learned.
Comments
Anonymous
March 18, 2010
Hi Ken, "foolish statements" - I agree. Every time I read your blog and book, I learn something new. I made some testing on the MSDN forums sign in system and you can’t imagine some surprises I found: http://speechflow.spaces.live.com/ Thanks! Javier Andrés Cáceres AlvisAnonymous
March 23, 2010
Dear Ken, This is one of the best and most comprehensive posts I've read on the entire Toyota issue - if only there was more common sense in the world. thankyou.Anonymous
March 23, 2010
Hi Ken, I also wanted to add that telemetry is something which will be adopted by toyota soon.