Reliable Intent
I've been super busy talking with as many testers and leads on my team as I can, gathering data on what is and is not working for them. I'm completely unsurprised to learn that, as with many other teams I've talked with, they're having problems understanding what tests mean to test, finding and using functionality from other areas, and getting reliable results from tests.
Over the course of these discussions, everything I heard started grouping into two buckets: Intent and Reliability:
- Intent: Our tests contain only the details that matter.
- Reliability: Test failures always mean a product bug.
Intent harks back to the stack I helped build on Expression, although I'm finding my thinking around that is evolving (more details soon!). Reliability will come in part from reorganizing our current stack and merging our multiple ways to do <whatever> into exactly one that lives in an easily discoverable location. Beyond that, we're going to put some metrics around reliability to help us identify the most problematic functions, make them reliable, then move on to the next set. I'm sure we'll find problems in our test code, in our product code, in other teams' product code, and in the underlying device code. We have good working relationships with all of these groups, and a history of working through similar problems with them, so I know we'll get everything straightened out.
In related news, yesterday I purchased a data plan for my WP7 phone, so now I can read and respond to email even when I don't have wifi. I don't think I'm turning into a phonehead though - I haven't yet found a situation where I wouldn't have rather had a tablet I could ink on rather than use a tiny keyboard that makes my fingers hurt. Heeeeeeeeeeeere, tablet tablet tablet tablet tablet! (I hope I get a shark!)
In separate related news, I'm updating my You Are Not Done Yet checklist with mobile-related items and topics. If you have any, let me know!