Flow-Driven Development for Agile Teams

I’ve been involved in many agile projects and practiced agility on personal projects. However, sometimes, I feel there are still practices in our process which are not really aligned with the agile concept. I believe there is a lot we can do to see the real capabilities of agile development. Everything I am talking about here is in the context of Scrum, but may apply to other frameworks as well.

Problem

To build something you need to have a sketch in mind, build it and test it to see how it works. That is almost the same story with developing software; it requires three activities regardless of the methodology being used: design, implementation, and testing. However, different methodologies utilize these activities in different ways. Some scrum developers still use "mini waterfalls" to develop user stories in each sprint and this mostly happens when a developer shifts from waterfall mindset to Agile. “Agile Development” tries to deliver shippable features in each sprint and be more focused on user stories and reacts to the changes. The sequential development process limits the agility of the development in a sense that a design should be at least partially completed to start implementation (don’t tell me you are one of those who jumps into implementation with absolutely no design at all!) and an end-to-end testing is not possible until a big piece of implementation is done, let’s say half way through the sprint. By testing I mean making sure that from the beginning to the end of the user story every piece of code works well and matches what user expects. Halfway through the sprint, what if the developer finds out that the implementation does not quite match the user story after running the end-to-end scenario for the first time? This is analogous to building different pieces of a car separately and then assembling them at once. The car might work well, but we could find out that it does not look good with the big bumpers or it is not comfortable because we did not realize that the dashboard is too big comparing to the whole interior space. This is the result of getting too deep into one piece before sketching the whole thing. We would have been able to see these issues if we’d had the “so-far” image of the car in mind during the building process or stepped back once in a while and took a look at the overall progress and the result. My main point is that with the current development process, the end-to-end scenario, which I call it “the flow” of the user story, does not come together till the near end of the implementation.

I believe agile development needs an interactive and flexible development practice that enables developers to focus on the essential development activities, customize the sequence of developing different pieces for each user story, change the design and implementation very easily, and most importantly start the end-to-end testing from the very beginning of the development. It is very crucial that the flow of the user story in the code matches what the users expect. It is very easy to get caught in the implementation details and lose the big picture.

In addition, we all have seen that documentation is not a priority for hardcore agile developers; however, I strongly believe that investing in “proper documentation” will pay off in the long run. Sometimes documentation is part of the deliverables and very clear how it should look like, but if it is not being delivered, the "proper documentation" should enable somebody new to the team or any team member after 2 years understands the system and be able to maintain it. But the truth is documents such are usually time-consuming to produce and get out of sync with the actual implementation over time.

I am proposing a practice in which you can address the above issues in your agile development environment. I am not trying to change how you design, implement, or test, rather just use a different order and introduce some steps in between.

I have tried this many times and gotten very good results so far. The practice could use more developers to adopt and lots of feedback (Thank you in advance J ). Since this development practice focuses on the end-to-end flow of the user story, I call it “Flow-Driven Development”.

My Solution, the Flow-Driven Development

Overview:

Flow-driven development is a development process suitable for developing user stories in an agile environment such as Scrum. Developers integrate the design, implementation, and testing of a user story into one comprehensive phase, consisting three activities: Stubbing, Filling, and Walkthrough. Each of these activities tries to accomplish the goal of a waterfall phase, i.e. design, implementation, and testing, but they could be practiced in different orders to overcome the problems I described before. I need to mention that the details of executing these activities are decided based on the team and the project dynamics.

Also please note that this process relies heavily on code comments which satisfy a specific format presenting all necessary information for development activities (which could be standardized).

The practice in a high level:

Having a user story in hand and a good-enough design in mind, “stubbing” starts from high-level items to low-level ones (top-down approach). I want to stress on the fact that you do not need to have all the high-level objects in your system laid out before starting the stubbing; start from a few ones and other stubs will be added gradually as the process moves forward. Stubs are first created for classes. I suggest putting the following comments at the top of each class: 1) the concept in the system that the class represents (e.g. student), 2) responsibilities of the class (e.g. take courses), and 3) relations/interactions to other classes (e.g. has a teacher in each course). Other comments may be required occasionally. Then, stubs are created for class members to enable the class to accomplish its responsibilities and provide proper communication channels and interfaces to other classes and the outside world. I would provide each member at least with the following comments: 1) the description of data/functionality that the member caries (e.g. the number of credits the student has taken so far), 2) for functions, the description and the valid range of input/output and any exception/error the function may throw, and 3) who should/may use this member. We should be able to follow the flow of the user story from the beginning to the end by reading the comments. Be sure to have the code compile-able and runnable at all time; for instance, all methods and properties that need to return something have a return value hardcoded for now. The user story may not be complete now, but we should be able to walk through “a happy path” from the beginning to the end.

When you feel that you have laid down the skeleton of the user story, the filling starts. Keep in mind that we want to have a happy path of the end-to-end scenario stubbed at the beginning before getting lost into the implementation details. Each stub is being filled accomplishing what is described in comments. The order of the filing is totally up to you, but I like to start from simple stuff, so I fill a method that verifies an email address before the one that calculates a company’s revenue. The filling activity contains two steps: the implementation is described in natural language at the top of the stub and then converted to the code.

A unit test needs to be written if applicable (and yes, it is worth the time; if you care about the quality, write the unit test). You can write the unit test just after the filing or utilize the TDD, whichever you are more comfortable with.

The filled stub should be tested by walking through the different scenarios of the user story that the stub is involved in. During the walkthrough, the developer does what the user would do, but he also can see what is going on under the hood. The developer must refer to the user story to validate the flow has been implemented precisely. The developer can change the flow anytime if it does not “feel” or “look” right!

Developers can go back and forth between different activities if needed. For example, a class member stub maybe created during filling another member stub to provide required data/functionality. The return value of this new member can be hard-coded for that moment just to get us going and can be filled later. Also a walkthrough can be done before any filling just to make sure the code flows between different components correctly. In my opinion, this is a very good “design review” practice!

These three activities are repeated until all the stubs are filled completely and walkthroughs are successfully passed on all paths of the user story. Obviously unit tests are all green at this moment. Also, at the end, a tool can extract all the comments from the code to form a document. The better and more complete the comments, the better the document.

In summary, using mini waterfalls to implement user stories in sprints may not be the best option and has its own limitations. Since user stories are a key part of the agile development, we want to make sure that the delivered product does exactly what the stories are telling. This goal can be achieved more effectively if we always have the end-to-end scenario in mind. By integration the three phases of the traditional waterfall methodology and paying more attention to the flow of the user story we have a better chance of making our customers happy.

Good day to you.

Hamed

Comments

• Anonymous
  July 19, 2010
  The comment has been removed
• Anonymous
  July 19, 2010
  The comment has been removed