The Test Test
I am always frustrated and somewhat sad when I hear testers whine or complain that they are not treated fairly; or that they are not respected; or that their development peers look down on them. I've been sitting on this post for many months wondering if I should post it or not when this thread popped up over on the JoS boards.
On one hand, I am always happy to offer words of encouragement and advice on how to rise out of the situation or at least to make the best of it. On the other hand, part of me sometimes wants to just say "stop your whining. If you don't like it that much, quit and find someplace to work where you will be treated fairly and be respected!"
If you want to find a good testing job, you just need to ask a few questions. That said, with all due respect and references to Joel and The Joel Test, I give you "The Alan Test"
The Alan Test - aka "The Test Test"
- Are testers influential from day one of the project?
- Does the test team own their own schedule?
- Does the test manager report to the general manager (and not to development)?
- Are career paths for testers and developers equal?
- Do the developers value testers?
- Do testers have the same working conditions and resources as development?
- Do testers use good test case management and source control tools?
- Are tests built daily?
- Are automated tests and manual tests valued appropriately?
- Do testers have the same coding guidelines and rules as developers?
- Is there a culture of quality?
Details:
Are testers influential from day one of the project?
Notice that I used the word influential and not involved or even hired. From day one of the project, testers should be reviewing specs, giving feedback on schedule, and driving testability. The full test team doesn't need to be on staff from day one, but someone should be there setting the quality bar early. If testers are not involved (or hired) before coding begins, the organization obviously doesn't value test (nor quality for that matter).
Does the test team own their own schedule?
The test team should own their schedule and have influence on the overall product schedule. A one-week code complete slip cannot be "absorbed" in the test schedule. If the test team determines they need n days or weeks after code complete to finish testing, they need n days or weeks. Period. If the test schedule is also known as "buffer for the dev team", the organization doesn't recognize the value of test.
Does the test manager report to a general manager (and not to development)?
Put another way, the test manager should be a peer of the development manager. If the test manager reports to the development manager, development needs drive test, and test has a lesser voice in the product.
Are career paths for testers and developers equal?
If test and development are indeed peers, they should have equal career paths. At Microsoft, we have "levels" that line up with promotions and career paths, and developers and testers have equal opportunity for promotion. Another way to ask this question could be "would you ever pay a tester as much as you pay your top developer?". Don't fall for the paper trick on this point - as in "On paper, testers can grow as much as developers - look, we have documents". Ask for examples. "How many developers and testers are at your most senior levels in your org". If the organization isn't willing to promote testers to senior positions, they don't value test.
Do the developers value testers?
Ask if the developers see test as an ally in creating quality software, or as a gang of hooligans making their lives hard (or as a bunch of robots pushing buttons)? Testers don't exist to make developers cringe or cry. In a good organization, developers understand that the role of test can be as much about quality assurance as it can be about quality control, and know that the test team exists so that everyone can make a higher quality product.
Do testers have the same working conditions and resources as development?
Would you want to work somewhere where developers had their own offices, dual 22 inch wide-screen monitors and comfortable chairs, while the test team worked in the hallway sitting on milk crates?
Me neither.
Do testers use good test case management and source control tools?
I once tested software on a laptop, in a car on the way to drop off the master for duplication (at least I wasn't driving). I'm a fan of ad-hoc testing, but this was over the line (I don't work at that company anymore).
Testing is a creative activity, but some structure around recording test cases and related test code is critical on a professional test team. A test case management tool is necessary - as is the ability to version test cases and test code. If the test team doesn't value this, they probably don't really care that much about testing.
Are both tests and product code built daily?
Automated tests (assuming they are compilable code and not scripts) should be built at the same time, and in the same process as product code. This is particularly important in situations where the test code calls APIs or other functionality in the product code, as it provides a small level of testing at build time (function signatures and other header resources).
Do testers have the same coding guidelines and rules as developers?
Put another way - Is test code treated the same as product code. If the test team is writing unmaintainable buggy code, you could hardly expect the development team to respect them. Test code is just as important as production code, and is in need of similar efforts.
Are automated tests and manual tests valued appropriately?
Does management have unrealistic goals of test automation, or do they devalue all manual testing? Unrealistic goals for automated tests indicate that management doesn't understand testing well enough for you to want to work there. Similarly, if all automation is de-valued by management, this indicates that management doesn't understand testing well enough for you to want to work there. Ask about the product and testing goals, then ask how automated and non-automated tests support those goals. Ask for examples of tests on that team that are automated and for examples of manual tests. If a team tries to automate too much - or not enough, it's a sign that you probably don't want to work there.
Is there a culture of quality?
Finally, you need to determine if quality is something the organization tries to test into the product, or if it's something that drives everyone on the team. Do developers "throw code over the wall" to test, or are they embarrassed and apologetic when bugs are found? Are bugs fixed as they are found, or are they left to fix at the end. In order to meet schedule, are bugs punted, or are features cut?
Eleven simple questions. Eleven questions where I would bet the majority of the answers for many testers are "no". I wouldn't work in an organization that scored less than 9. Sadly, many organizations are much, much lower.
Of course, if you know of an eleven, please let me know where to send my resume. :}
Comments
- Anonymous
July 31, 2008
Thanks for the story, after I read it I can say without reservations that I am at the right place. Your assessment equals that of myself (The test manager) and the Sr. Management.Great post, I will refer other to it in the future. - Anonymous
July 31, 2008
Ouch, my last company would have scored ZERONo wonder I leftWould be interesting to do a survey and see what scores you get and what the average is - Anonymous
August 01, 2008
Thought-provoking post! I think that your questions are also relevant where testers and development belong to/ are different organizations.It might be possible for a tester to influence the conversion of a negative answer to positive. Presenting one's case well with data to substantiate the case may help this process along.Thank you. - Anonymous
August 01, 2008
This is a great list. I would want to add one more thing to your question 5 ("Do developers value test?"):Do testers value developers?You ask the question in the inverse, and I think it's important, but the walls and the general angst about test in an organization can be built by the organization..... and also by test. There's an old adage that when there's a problem between two people, it's never 100% one person's fault. I think that's the case here as well: the tester's attitude can create failure, just like the organization's attitude can create failure.As usual I had more to say than would fit in a comment, so I put it here: http://blog.abakas.com/2008/08/dont-be-part-of-problem.html - Anonymous
August 04, 2008
hmm,i am getting almost 3 , but as a Performance contractor hardly these points makes any difference unless you work for companies like microsoft. - Anonymous
August 04, 2008
Thank you Alan for a terrific post. - Anonymous
August 04, 2008
Thanks Jake - I haven't heard from you in a while, and it's nice to see you're still reading my blog.Catherine - thanks for the comments (and the follow up post). I've added your blog to my blog roll (let that be a hint to any other blogers who read my blog - if you post a comment, I'll read your blog!).It's purposely a tough test, and I'm not surprised to hear of folks with low scores. But I will tell you that there are 9's and 10's out there if you look hard enough, and that putting in the extra effort into finding a 9 or 10 is absolutely worth it. - Anonymous
August 04, 2008
Follow up question to your comment above - what score was Microsoft when you joined and did you give them "The Alan Test" ? - Anonymous
August 05, 2008
Great question Philk. Before I answer, I'd like to point out that even today, there is variation in scores between teams at Microsoft.Also keep in mind that I started at ms over 13 years ago (probably the point of your question). Anyway - I joined the networking team on win95. Our team was a 3 perhaps a 3.75 because about 3/4the developers valued the test team.I don't get to do much testing on the EE team, but my last team was a solid 8 working toward a 10. - Anonymous
August 05, 2008
so there's an idea for the next blog, how to go from 3 to 8 and 8 to 10... - Anonymous
August 05, 2008
Every once in a while, I write a post that gets a comment - my last post (the Test Test ) had half a - Anonymous
August 07, 2008
(thanks to my colleague, John, for the Spinal Tap match - it's perfect even if not every test team at - Anonymous
August 14, 2008
More tips on how to go from 1-11 on the Test Test . Test #2 on the Test is " The test team owns - Anonymous
August 17, 2008
Not a physical place - but an organizational place. Test #3 of the Test Test says that the test manager - Anonymous
August 25, 2008
Test #5 of the Test Test asks if the development team values the test team. Catherine pointed out that - Anonymous
August 28, 2008
I remember visiting a company once where the testers were lined up at desks along the wall...in a long - Anonymous
September 02, 2008
Test #7 of the Test Test asks if testers use test case management and source control tools. The only - Anonymous
September 06, 2008
I think I can lump some ideas of how to improve #8 and #10 (Are tests built daily, and does test code - Anonymous
September 09, 2008
The 11th (and final) question in the Test Test asks if the organization has a culture of quality. Does