Acceptance tests with BDD - Does it make sense?

We are repeatedly asked how the use of BDD should be assessed. At first glance, many reasons speak in favour of Behaviour-Driven Development:

• Readable test cases are created.
• This goes hand in hand with a high degree of traceability and transparency of the test cases.
• There is a „single source of truth“ for the requirements to be implemented.

So why shouldn't we use BDD everywhere? Or to put it another way: Does BDD fit into every project's daily routine?

The fundamental idea of BDD

BDD is a Developmentprocess, which can be reduced to four steps:

1. Before a new feature is integrated into software, the requirements are clarified. A formalised syntax is used for this, e.g. Gherkin (Given-When-Then format) or Markdown.
2. Once the implementation begins, the (formalised) requirements will be incorporated into test automation. The tests will initially FAIL because they describe behaviour that has not yet been implemented in the software.
3. Subsequently, the functionality and, in parallel, test automation will be implemented. The test cases will be continuously executed during this phase.
4. At the end of this cycle, the requirements will have been successfully tested.

At its core, it's therefore a test-driven approach that formal requirements at the beginning of the development process.

In practice, lifecycle management systems can be used where scenarios are executable for both manual and automated test cases. For example, with OpenText ALM Octane or in Jira via X-ray the scenarios captured directly in Gherkin syntax as test cases.

Requirements as a connecting element

These (formalised) requirements form a connecting unit – from business, development, QA, to infrastructure teams. Each team contributes its perspectives and thus develops a common data foundation.

BDD therefore requires high-quality requirement engineering and a cross-functional team.

In practice, however, other constellations are repeatedly encountered:

• The development and QA teams are organisationally separate and are also involved in the project in a downstream capacity.
• Requirements are formulated by separate business analysts, often in free text and in tools that are not accessible to QA or infrastructure teams.

It is difficult to make a general statement about the sensible use of BDD here.

Those who wish to live BDD should therefore consciously ask themselves the following questions:

1. Is there a cross-departmental development team?
2. Is it justifiable to gather formalised requirements solely for test automation?

BDD and Test Automation

When looking at common tutorials on BDD, one can quickly get the impression that Gherkin is an easy-to-use language.

She is too – if you stick to a few rules of the game:

Every requirement, every scenario, is test automation source code.

• The well-known Clean Code principles should apply, e.g. „Don’t Repeat Yourself“ or „Keep It Simple, Stupid“.
• Each requirement should typically be 5-7 lines long – no more!
• At regular intervals, a review and refactoring of the scenarios should be carried out.

Every requirement, every scenario, describes expected behaviour.

• UI designs or UX designs should be in separate documents, not in the scenarios.
• Instead, the workflow of all artefacts should be described:
  • What is the initial state?
  • What's happening to them?
  • What do the artefacts at the end of each process step look like?

3. Each requirement, each scenario, must be (continuously) testable.

• BDD does not solve structural problems in test automation.
• Test data or test clusters must be available and controllable via CI servers.
• In the frontend, stably recognisable objects must exist, and APIs must exhibit corresponding consistency.

BDD does not replace a test automation framework, but rather adds another layer.

BDD and acceptance tests

In acceptance testing, comprehensive workflows are generally tested. The test cases often comprise many individual steps, which are in themselves complex to automate. Even without BDD, this testing level is the most time-consuming process step within product development. Does this, conversely, mean that BDD should be foregone at this level?

As is often the case, it helps to look at the entire test process. If each individual process step can be tested automatically, what is the remaining risk in the overall process?

The focus on the process steps helps in several ways:

• Test data can be provided for each step to also test special constellations.
• In each step, exceptional situations (connection interruptions, program crashes, etc.) can be simulated much more easily.
• If individual process steps are changed, this only affects a subset of all tests. Therefore, it is not necessary to adjust all tests as a precaution.

„Acceptance testing“ can therefore be highly focused. Exploratory tests or A/B tests with selected user groups are often more cost-effective here and cover the same risk as attempting full test automation with BDD.

A general answer on whether BDD is useful for acceptance testing is ultimately only possible within the specific project context.

Summary

BDD connects all professional fields based on unified, formalised requirements.

The use of BDD is not a foregone conclusion and requires careful planning on the project. Just because the syntax appears modern and simple, it still requires continuous effort to use this level in test automation meaningfully.

Podcast recommendation

I was a guest on Richie Seidl's podcast „Software Testing – Quality, Test Automation & Agility“. In the episode „A Critical Look at BDD“, I discuss with the host why BDD isn't always the best solution.

This way for the episode.