Skip to main content

Handling work-in-progress with Thucydides and JBehave using @pending and @wip tags

Posted by johnsmart on August 20, 2014 at 8:36 AM PDT

Thucydides version 0.9.268 has just been released, with a few very interesting new features. Thucydides is an open source reporting library that helps you write more effective BDD-style automated acceptance criteria, and generate richer test reports, requirements reports and living documentation. In this article, we will look at some of the new ways this version lets you handle work-in-progress or pending scenarios with Thucydides and JBehave.

In JBehave, a scenario is considered passing if all of the step definitions are implemented, even if there is no code. This is because there is no obligation to use step libraries within the step definitions, though it is a good practice for more complex tests. Consider the following scenario:

Scenario: Logging on via Facebook
Given Joe is a Frequent Flyer member
And Joe has registered online via Facebook
When Joe logs on with a Facebook token
Then he should be given access to the site

When you execute this with no step definitions, it will be reported as Pending, as illustrated here:

pending_steps

When you implement the steps, they will be considered successful unless an exception is thrown or a step is marked as pending. So the following will indeed pass:

    @Given("$username has registered online via Facebook")
    public void has_registered_via_facebook(String username) {}

This is because there is no way to know that a step definition is empty - we can only know that no @Step methods were called, which does not necessarily mean that it is empty.

You can make this a pending step by using the org.jbehave.core.annotations.Pending pending annotation, e.g:

    @Pending
    @Given("$username has registered online via Facebook")
    public void has_registered_via_facebook(String username) {}

JBehave and Thucydides will now report this scenario as pending, even though it has an "implemented" (albeit empty) step definition:

step-details

This is also a good way to keep track of work if you are driving the code from the step definitions, as you can easily see which steps have been done at any point in time.

The @ignore tag lets you skip a story during test execution, so that it does not appear in the reports.

Meta:
@ignore

Scenario: Logging on via Facebook
Given Joe is a Frequent Flyer member
And Joe has registered online via Facebook
When Joe logs on with a Facebook token
Then he should be given access to the site

If you want a scenario to appear in the report, but to mark it as 'pending' even if it fails, you can use the @pending tag directly within the story files, e.g.

Meta:
@pending

Scenario: Logging on via Facebook
Given Joe is a Frequent Flyer member
And Joe has registered online via Facebook
When Joe logs on with a Facebook token
Then he should be given access to the site

Scenario: Logging on via Twitter
Given Joe is a Frequent Flyer member
And Joe has registered online via Facebook
When Joe logs on with a Facebook token
Then he should be given access to the site

or, for an individual scenario:

Scenario: Logging on via Facebook
Meta:
@pending

Given Joe is a Frequent Flyer member
And Joe has registered online via Facebook
When Joe logs on with a Facebook token
Then he should be given access to the site

In this case, the entire scenario or story/feature will be reported as 'pending':

Screen Shot 2014-08-20 at 4.48.10 pm

You can also distinguish between work that hasn't been started yet and work that is in progress but not yet complete. The @skip or @wip tags will act like the @pending tag, but will report the scenario or story as "skipped".

Scenario: Logging on via Facebook
Meta:
@wip

Given Joe is a Frequent Flyer member
And Joe has registered online via Facebook
When Joe logs on with a Facebook token
Then he should be given access to the site

These will appear differently in the reports, as shown here:

 Screen Shot 2014-08-20 at 5.10.59 pm

This is a good way to identify what features are currently being worked on.

 The following table summaries these options:

What Where Outcome
@Pending annotation Step definition code Individual step is flagged as 'pending'
@pending tag Scenario metadata in the .story file The whole scenario is flagged as 'pending'
@pending tag Story metadata in the .story file All the scenarios in the story file are flagged as 'pending'
@skip or @wip tag Scenario metadata in the .story file The whole scenario is flagged as 'skipped'
@skip or @wip tag Story metadata in the .story file All the scenarios in the story file are flagged as 'skipped'
@ignore tag Story or scenario metadata in the .story file The story/scenario will not be executed and will not appear in the reports

 

John Ferguson Smart is a specialist in BDD, automated testing, and software life cycle development optimization, and author of BDD in Action and other books. John runsregular courses in Australia, London and Europe on related topics such as Agile Requirements GatheringBehaviour Driven DevelopmentTest Driven Development, andAutomated Acceptance Testing.