Category Archives: BDD

Business readable progress and flow based on acceptance tests improves collaboration and trust

A couple of weeks ago I was attending Agile 2011 in Salt Lake City, where I held a brief 8 minute lightning talk about how we made progress and team flow in our projects business readable and what benefits we gained out of it. I want to summarize this talk with the following blog post.

Tasks are not business readable

So, what does business readable progress and team flow mean? Typically, teams break down user stories into task, in order to self-organize their work in delivering the story. Unlike user stories, tasks are not necessarily understandable to business:


Picture: the teams breaks a user stories into tasks (<8h) to self-organize their work

Business stakeholders can just see on the task board, when all tasks of a user story are done, and based on this information assume that the story is done. Individual pending or completed tasks usually don’t tell outsiders what is working already and what is not – sometimes this is even not obvious to other team members not working on this particular story.

Planning work and limiting work-in-progress based on tasks

When we started using SpecFlow, our workflow of preparing a user story for implementation looked like this:

  1. Collect examples and artifacts related to the user story: samples or existing documents, lo-fi UI scribbles, etc.
  2. Have a discussion around these artifacts and examples, and identify individual acceptance criteria that are illustrated with according scenarios using examples.
  3. Team commits to build the story based on the discussed acceptance criteria and scenarios.
  4. Team performs task planning for the committed user stories (we are working in sprints, but you could as well have a WIP and size limit for continuously pulling user stories into implementation).
  5. Team implements individual user stories based on tasks, starting with formalizing the scenarios into Gherkin for automating them. They organize work along the planned tasks. Actually, formalizing scenarios into Gherkin are tasks themselves. And of course, tasks are refined and adjusted as needed, as they story is implemented.

This approach had several problems:

  • Tasks spanned multiple acceptance criteria. User stories often fulfilled no single scenario until the very end, when the final tasks of a story were completed.
  • Sometimes the team dealt with formalizing and automating the Gherkin scenario only late, when the user story was already (almost) done. This caused frictions in automation (making an already implemented feature “automatable” in the end).
  • Delaying formalization and automation after task planning or even until the story was almost done brought up important discussions at a time when it was already rather late to consider the conclusions. Also, tasks that were planned before the formalization of scenarios became void as new things were discovered during formalization.

Planning work and limiting work-in-progress based on scenarios

In our retrospectives we tried to address those problems and defined the following goals:

  • Finish individual scenarios earlier, ideally one scenario of a user story after each other
  • Delay planning of tasks further to take important conclusions into account

After several experiments, we arrived at a workflow, where we formalized the Gherkin scenarios beforedoing the task planning. Formalization was done in pairs within the team, while task planning was still a collaborative effort for the whole team. During formalization, new questions that came up could be clarified with other project stakeholders as needed. The formalized scenarios were reviewed by the whole team during task planning:


Picture: list of scenarios discussed for a user story


Picture: one scenario formalized into Gherkin

This workflow had the following positive effects on the flow of implementation:

  • It was easier for the team to plan tasks, and fewer tasks became irrelevant during implementation.
  • Tasks aligned with individual scenarios which allowed it to limited work-in-progress on tasks for a specific scenario. As a result, scenarios for a given story could be fulfilled one after another.
  • Some teams even managed to fulfill individual scenarios in less than a day, and dropped additional task planning completely. Instead they just reviewed the formalized Gherkin scenarios together, before starting with the implementation.

Business readable progress yields faster feedback

After establishing this improved implementation flow, we discovered further benefits. To explain these, I need to describe our continuous integration setup:

After each check-in, our build server runs all automated scenario tests and deploys the new build to a staging system. Actually, we have two kinds of builds:

  • One build is running the scenarios for stories currently in implementation only. This build runs on every check-in.
  • The other build is running all automated scenarios of all stories completed already. As running all these tests takes quite some time, we are running this build only when a previous build has not been started since a certain period, to avoid having multiple instances of the build running in parallel on our build server.

Along with each deployed build, there is a test report, which lists all specified scenarios and their execution results:


Picture: test report listing all user stories committed for implementation as features, along with the execution results of their scenarios

  • Pending (purple): the scenario has not been automated yet
  • Failed (red): the scenario is automated and not fulfilled by the system
  • Success (green): the scenario is automated and fulfilled by the system

The deployed builds are posted along with their test report to a project dashboard, where all stakeholders can access them easily:


Picture: Project dashboard showing work-in-progress, including currently deployed build and automated scenarios report

This approach made the progress and flow of the team business readable. Instead of waiting for all tasks to be completed, stakeholders can now track completion of individual scenarios. They can see which user story is already working in a given deployed build, to start testing and give feedback.

Since user stories are implemented scenario after scenario, testing and giving feedback can start even earlier, when the core scenarios of a story are working already. Additional scenarios, for example extended validation or optional flows to be supported, can be tested later.

Better involving business into the BDD cycle

In our previous approach, business reviewed formalized Gherkin scenarios only very late – partly because they were completed late. It was sometimes even hard to keep business interested in the formalized scenarios, after we had the initial discussions for planning them.

Improving transparency and shortening the feedback cycle helped us to keep business and stakeholders more involved during the implementation. Conversations around the scenarios continued until the story was finally completed. This closer collaboration also increased the trust among all parties.


In summary our approach to make progress and team flow business readable provided us the following benefits:

  • Deferring task planning after formalization of Gherkin scenarios improved planning quality and aligned tasks with scenarios. Some teams even dropped task planning and organized their work only based on formalized scenarios.
  • Aligning tasks with scenarios allowed the team to limit work-in-progress on a scenario level, finishing scenario after scenario of a given user story. Stories could be tested already with a sub-set of scenarios, before the story was fully completed.
  • Providing transparency about already completed scenarios allowed earlier testing and feedback within the team as well as for business stakeholders.
  • Shortening the feedback cycle improved collaboration with business and helped building trust.

Further references

  • The agile alliance website hosts description and slides of my presentation.
  • We are using SpecFlow to write automated acceptance tests that are business readable.
  • We are mapping and visualizing SpecFlow scenarios using SpecLog.