Trying out Behave#
One of the neat things I saw at the Agile conference was a short demo of RSpec and RBehave. Intrigued, I did a quick search and found the .NET equivalents: Behave# and NSpec.
Note: I'm not sure I completely understand the differences between xSpec and xBehave as they both quote Behaviour-Driven Design as their goal. I think that the Behave libraries are just a way to change the nomenclature of xUnit from 'test' to 'behaviour'. The Spec ones are about writing user stories in a sort of DSL that a product owner or PM could understand.
Since developer stories can be user stories, I thought it would fun to use Behave# to implement the Stack example from Jim Newkirk's Test-Driven Development in .NET book. Here's what I ended up with (apologies for the formatting - it's much easier to read inside VS):
Stack stack = null;
bool isEmpty = false;
object tempObject = null, tempObject2 = null, tempObject3 = null;
List<int> knownObjects = new List<int>();
knownObjects.Add(1);
knownObjects.Add(2);
knownObjects.Add(3);
Story stackStory = new Story("Basic stack operations");
stackStory
.AsA("developer")
.IWant("to peform operations with a stack")
.SoThat("I can save my data");
stackStory
.WithScenario("A brand new stack")
.Given("a stack I created", delegate { stack = new Stack(); })
.When("I query its state", delegate { isEmpty=stack.IsEmpty; })
.Then("it should be empty", delegate { Assert.True(isEmpty); })
.Given("a stack I created")
.When("I push a single object",
delegate { stack.Push(new object()); })
.And("I query its state")
.Then("it should not be empty",
delegate { Assert.False(isEmpty); })
.Given("a stack I created")
.When("I push a single object")
.And("pop the object", delegate {tempObject=stack.Pop(); })
.And("I query its state")
.Then("it should be empty")
.Given("a stack I created")
.When("I push a known object",
delegate { stack.Push(knownObjects[0]); })
.And("pop the object")
.Then("the two objects are equal",
delegate {
Assert.Equal<object>
(knownObjects[0], tempObject); })
.Given("a stack I created")
.When("I push three known objects",
delegate{knownObjects.ForEach(
delegate(int val){stack.Push(val);});})
.And("pop each one ",
delegate { tempObject = stack.Pop();
tempObject2 = stack.Pop();
tempObject3 = stack.Pop(); })
.Then("all three objects are the same",
delegate
{
Assert.Equal<object>(knownObjects[2], tempObject);
Assert.Equal<object>(knownObjects[1], tempObject2);
Assert.Equal<object>(knownObjects[0], tempObject3);
})
.Given("a stack I created")
.When("I pop an object it throws an exception",
typeof(InvalidOperationException),
delegate(Type type)
{ Assert.Throws(type, delegate { stack.Pop(); }); })
.Then("", delegate { ;})
.Given("a stack I created")
.When("I push a single object")
.And("call top", delegate { tempObject = stack.Top(); })
.And("I query its state")
.Then("it should not be empty")
.Given("a stack I created")
.When("I push a known object")
.And("call top")
.Then("the two objects are equal")
.Given("a stack I created")
.When("I push three known objects")
.And("call top")
.Then("the last item is equal to the top one",
delegate
{ Assert.Equal<object>(knownObjects[2], tempObject); })
.Given("a stack I created")
.When("I push a known object")
.And("call top repeatedly",
delegate
{
tempObject = stack.Top();
tempObject2 = stack.Top();
tempObject3 = stack.Top();
})
.Then("all objects are equal to what was pushed",
delegate
{
Assert.Equal<object>(knownObjects[0], tempObject);
Assert.Equal<object>(knownObjects[0], tempObject2);
Assert.Equal<object>(knownObjects[0], tempObject3);
})
.Given("a stack I created")
.When("I call top it throws an exception",
typeof(InvalidOperationException),
delegate(Type type)
{ Assert.Throws(type, delegate { stack.Top(); }); })
.Then("")
.Given("a stack I created")
.When("I push a null object", delegate { stack.Push(null); })
.And("I query its state")
.Then("it should not be empty")
.Given("a stack I created")
.When("I push a null object")
.And("pop the object")
.Then("the returned object is null",
delegate { Assert.Null(tempObject); })
.Given("a stack I created")
.When("I push a null object")
.And("call top")
.Then("the returned object is null");
}
and when I run the tests, this is the output:
Story: Basic stack operations
Narrative:
As a developer
I want to peform operations with a stack
So that I can save my data
Scenario 1: A brand new stack
Given a stack I created
When I query its state
Then it should be empty
Given a stack I created
When I push a single object
And I query its state
Then it should not be empty
Given a stack I created
When I push a single object
And pop the object
And I query its state
Then it should be empty
Given a stack I created
When I push a known object
And pop the object
Then the two objects are equal
Given a stack I created
When I push three known objects
And pop each one
Then all three objects are the same
Given a stack I created
When I pop an object it throws an exception: System.InvalidOperationException
Then
Given a stack I created
When I push a single object
And call top
And I query its state
Then it should not be empty
Given a stack I created
When I push a known object
And call top
Then the two objects are equal
Given a stack I created
When I push three known objects
And call top
Then the last item is equal to the top one
Given a stack I created
When I push a known object
And call top repeatedly
Then all objects are equal to what was pushed
Given a stack I created
When I call top it throws an exception: System.InvalidOperationException
Then
Given a stack I created
When I push a null object
And I query its state
Then it should not be empty
Given a stack I created
When I push a null object
And pop the object
Then the returned object is null
Given a stack I created
When I push a null object
And call top
Then the returned object is null
1 passed, 0 failed, 0 skipped, took 1.15 seconds.
So is it useful? Like most things, there are situations where it's appropriate and others where it's not. I'm starting with a contrived example to begin with, and so not demonstrating its full usefulness. Regardless, there are a few points I'd like to make after playing with it for a while:
- It's nice to repeat snippets of an English sentence instead of code, although I suspect the novelty would fade after a while :)
- If you want to specify Exceptions as part of your story, this forces you to move away from [ExpectedException]. Which is a Good Thing anyway. I'm using an Assert replacement from Codeplex to accomplish this.
- I'm a big fan of having very descriptive method names for unit tests - this is taking descriptive to the next level :)
On the (possibly) less good side of things:
If a story fails, then the rest of the test isn't executed. It doesn't make sense to only put one scenario per test either, because then you'll end up repeating the delegate code.
If a story fails, it's sometimes really awkward to figure out why. It's really annoying when you have to debug into the delegate.
The test/stories are dependent on each other. This is a very disconcerting concept coming from an EDD background.
It's very difficult to test loops, or perhaps I just haven't found a good way. If you look at the place where
Top()is called repeatedly (fifth last story) and we check to make sure it always returns the same value, I had to use named objects to accomplish this. The original test looked like this:[Test]public void PushTopNoStackStateChange(){ string pushed = "44"; stack.Push(pushed); for(int index = 0; index < 10; index++) { string topped = (string)stack.Top(); Assert.AreEqual(pushed, topped); }}and I would claim that's easier to read. There's some flexibility lost there.
I'd like to see some more real world examples of this being used; this could be a powerful tool in the right circumstances. And of course as the library is enhanced, some of the things I've talked about may become more feasable.
Comments
- Anonymous
August 30, 2007
Just had a look at your bheave# example and read a bit more about it. I agree with one of your point that if you want to specify Exceptions as part of your story, this forces you to move away from [ExpectedException]. Which is a Good Thing anyway.It is quite verbose thoughI think i am going to try a real world example and see how it goes or to put it more appropriately to see if it grows on me and how comfortable i feel. - Anonymous
August 30, 2007
I agree with Brad and Jim that we should be moving away from [ExpectedException] regardless of what framework we're working in :)The verbose-ness (I think) is one of the qualities that could make it attractive to product owners or PMs. The code is really to express acceptance tests rather than developer-focused 'unit' tests.Please write back whenever you get a chance to try this in the real world .. I would love to see more examples :) - Anonymous
August 30, 2007
I wonder, and I'm allowed to be wrong on this, if it wouldn't be more effective to A) improve our programming languages (and usage of) for readability and B) require PM's to learn these languages. "But a PM would never learn...." it seems a lot easier (and cheaper) than forcing developers to learn and maintain this overly verbose artifact. - Anonymous
August 30, 2007
A) YesB) YesI guess in the meantime, it's interesting to see what we can accomplish with the tools we have :)Re the 'developers learning/maintaining stuff' .. you can bet that there is already work going on to do the translation from spoken language wiki documents (à la Fitnesse) to what I'm demonstrating here and more. - Anonymous
September 02, 2007
Esplorando il Behaviour Driven Development - Anonymous
September 03, 2007
The comment has been removed - Anonymous
September 03, 2007
Hi Joe,Thanks for the comments :) I'll definitely let you know how it works in the real world. It would also be really great to get some different (and more complex) examples than the Account one. - Anonymous
September 04, 2007
I couldn't agree with you more! I am getting tired of using the accounting example. I was thinking of something like a role playing game but that doesn't really apply to a majority of the vertical markets out there. May go with the traditional Video Store example.Nelson Montalvo has a 5 part series on using Behave#; he is using it to build the application he is working on.http://codemonkey.nmonta.com/2007/08/14/behave-part-5/JO