A small unit test primer (part two)

For some reason on the drive home yesterday, I realised there were a few more important points about unit tests (examples) worth mentioning.

• They should be fast. I'm talking Civic-Hatchback-with-a-Type-R-sticker-on-the-bumper fast.
• They should be independent. There is no guarantee that the tests will run in the same order each time, and that's OK. It also leads directly to the next point.
• They should not alter the machine state. After a test is complete the system should be in the same state as it was before the run.
• They should not depend on external services (where possible). I'm talking about web servers, network connections, Bob's Hooters & Google Maps mashup web service, etc.
• They should be consistent. Either they always pass, or they always fail, and neither the time of day nor position of the stars in the sky should have any effect on them whatsoever.

OK, enough soapboxing for one day.

Comments

• Anonymous
  November 21, 2006
  I certainly agree with your first four precepts and I am mostly in agreement with your final one. I do, however, have a caveat.You are very close to my caveat when you say "... and neither the time of day nor position of the stars in the sky should have any effect on them whatsoever." because when I am writing date-sensitive tests, I quite often throw in a test for tomorrow or next week. The reason that I do this is that there is a chance that as my tests are run in the future (this works best when you have a continuous build cycle), I may get an early warning of a problem tomorrow or next week. I have been saved more than once by this form of 'pre-emptive' testing.

• Anonymous
  November 21, 2006
  Time-sensitive unit tests are indeed important, and as a matter of fact I had to write quite a few myself earlier this year.  Because I was using examples (tests) to drive the design, and because I didn't have to deal with legacy code, I was able to implement my routines from the ground up to use a 'reference' time that got passed as a parameter. That way, all time-sensitive calculations could be taken care of in unit tests as well. For instance, I'd have fixtures that would use a 'standard' reference, like January 21, 1979 (for example), and then test +1 day, +1 week, next weekend, last week, etc.I think the method I've mentioned above covers your 'pre-emptive' testing, but it may not be possible to do that in your situation. My biggest concern with checking in unit tests based on the current system time is that if they break, depending on your team's environment, it may prevent everyone else from checking in code until they're fixed.

• Anonymous
  November 22, 2006
  There is no such word "Independant". It is spelled "independent". Nice post!

• Anonymous
  November 22, 2006
  The comment has been removed

• Anonymous
  November 22, 2006
  Hello, I'm an experienced frameworker, and have somethin to say ;) Did you mean something like this?        [TestMethod]        public void CacheReferencedTest()        {            IKernelObject<string, string> q = null;            IKernelObject<string, string> p = Kernel<string, string>.Create("{21130017-F36E-4a11-9BFD-C3CA11B5F859}", "222");            IKernelObject<string, string> r = Kernel<string, string>.Create("{82F4B050-F54D-46cd-99C0-C821B9A1256E}", "333");            Assert.AreEqual(Kernel<string, string>.Cache[p], p);            Assert.AreEqual(Kernel<string, string>.Cache[r], r);            q = Kernel<string, string>.Create("{21130017-F36E-4a11-9BFD-C3CA11B5F859}", "222");            q.References.Add(p);            p.References.Add(q);            q.References.Add(r);            Assert.AreEqual(r.Referenced[q], q);            q.References.Remove(r);            Assert.AreEqual(r.Referenced[q], null);            q.Referenced.Add(r);            Assert.AreEqual(r.References[q], q);            q.Referenced.Remove(r);            Assert.AreEqual(r.References[q], null);            Assert.AreEqual<IKernelIdentity<string, string>>(Kernel<string, string>.Cache[q], q);            Assert.AreEqual<IKernelIdentity<string, string>>(Kernel<string, string>.Cache[p], q);            Assert.AreEqual<IKernelIdentity<string, string>>(Kernel<string, string>.Cache[q], p);            Assert.AreEqual<IKernelIdentity<string, string>>(Kernel<string, string>.Cache[p], p);            Assert.IsTrue(Kernel<string, string>.Cache[q] == q);            Assert.IsTrue(Kernel<string, string>.Cache[p] == q);            Assert.IsTrue(Kernel<string, string>.Cache[q] == p);            Assert.IsTrue(Kernel<string, string>.Cache[p] == p);            Assert.IsTrue(q.State == KernelState.Cached);            Kernel<string, string>.Remove(q);            Assert.IsFalse(q.State == KernelState.Cached);            q.Add();            Assert.IsTrue(q.State == KernelState.Cached);            Kernel<string, string>.Cache.Remove(q);            q.Remove();            Assert.IsFalse(q.State == KernelState.Cached);            Kernel<string, string>.Add(q);            Kernel<string, string>.Add(q);            Assert.IsTrue(q.State == KernelState.Cached);            Assert.IsTrue(r.State == KernelState.Cached);            q.Referenced.Add(r);            Assert.AreEqual(q.Referenced[r], r);            Assert.AreEqual(r.References[q], q);            q.Remove();            q.References.Add(r);            Assert.IsTrue(q.References[r] == r);            Assert.IsTrue(r.Referenced[q] == null);            q.References.Remove(r);            Assert.IsTrue(q.References[r] == null);            Assert.IsTrue(r.Referenced[q] == null);            r.Referenced.Add(q);            Assert.IsTrue(q.References[r] == null);            Assert.IsTrue(r.Referenced[q] == q);            q.References.Remove(r);            Assert.IsTrue(r.Referenced[q] == q);            r.Referenced.Remove(q);            Assert.IsTrue(r.Referenced[q] == null);            q.Add();            q.References.Add(r);            Assert.IsTrue(r.Referenced[q] == q);            p.Referenced.Add(r);            Assert.AreEqual(q.References[r], r);            Assert.AreEqual(r.Referenced[q], q);            Assert.AreEqual(r.Referenced[q], p);            Assert.AreEqual(p.Referenced[r], r);            Assert.AreEqual(r.References[p], p);            Assert.AreEqual(r.References[p], q);            q.Remove();            Assert.AreEqual(Kernel<string, string>.Cache[p], null);            Assert.AreEqual(Kernel<string, string>.Cache[q], null);            Kernel<string, string>.Add(q);            q = p;            Kernel<string, string>.Add(q);            q.Remove();            q.Remove();            q.Add();            q.Add();            Assert.AreEqual(r.Referenced[q], null);            Assert.AreEqual(r.Referenced[p], null);            Assert.AreEqual(p.References[r], null);            p.Remove();            Assert.AreEqual(r.References[p], null);            Assert.AreEqual(r.References[q], null);            Assert.AreEqual(q.Referenced[r], null);            Assert.AreEqual(Kernel<string, string>.Cache.Count, 1);            IKernelStorage<string, string> backup = Kernel<string, string>.Copy(Kernel<string, string>.Cache);            r.Remove();            Assert.AreEqual(Kernel<string, string>.Cache.Count, 0);            Assert.AreEqual(backup.Count, 1);        }

• Anonymous
  November 23, 2006
  Yes ... sad to say I've seen 'unit tests' that looked like that before. Do I dare ask what product that code comes from?