Skip to main content

Does ATDD really save you time?

Posted by johnsmart on September 12, 2012 at 7:38 PM PDT

Acceptance Test Driven Development (ATDD) is a very effective development practice that essentially involves writing specifications in the form of documented and automated examples. These automated examples become automated acceptance tests that validate the features being delivered. The process of writing these examples encourages teams to focus on where the business value of a feature is coming from, which in turn helps developers aim for the most appropriate solutions in business terms.

When I help folks out with ATDD and TDD practices, one common question people ask me is this: Does using ATDD cost more in development time?

My gut feeling is that when you practice Acceptance Test Driven Development, developers spend maybe 20% extra time writing automated tests, compared to a project with no automated testing at all As far as TDD goes, I generally find that overall an experienced TDD practitioner will take similar or slighly less time to implement code using TDD than without using TDD.

However, that is far from the whole picture. Anecdotal evidence, and my own experience, suggest significant other advantages in adopting ATDD practices, and in fact projects using ATDD tend to deliver faster and with significantly fewer defectsthan projects using more conventional approaches. For example:

  • Writing specifications in the form of automated tests helps clarify the requirements earlier rather than later, and ATDD-style reporting also makes it very visible what is actully being delivered. This helps keep development efforts focused on building features that will provide real value, which in turn reduces waste.
  • Code written using TDD (especially using a BDD-style approach) not only has fewer bugs, but it tends to be much easier to understand and maintain, which makes changes easier (and less scary) to implement, and makes ongoing maintenance cheaper.

Automated Acceptance and Regression Testing (where automated tests are written after the features are delivered, but the tests do not play a driving role in the development process) are a great way to improve code quality and reduce the risk of embarrassing regressions and outages. However automated tests by themselves do not help focus and validate the development efforts the way ATDD does.

A recent study seems to confirm this. In this very interesting study, teams using agile development practices including in particular ATDD and TDD delivered projects 31% faster with 4 times fewer defects. Another study, back in 2008 found that using TDD reduced defects by between 40% and 90%, although it increased initial development time by 15-35%. The teams did agree however that this increase was offset by lower maintenance costs.

And yet another, from 2006, found that TDD significant improvements in the quality of the code (in terms of the defect rate), but were inconclusive with regards to productivity gains.

These studies are certainly very interesting. However I think that the earlier ones actually tend to understate the advantages of TDD, for two reasons:

  • They are often done with teams that were realtively new to TDD practices: However like many practices, TDD works better when you have already had some practice. In my experience, time spen
  • They do not factor in the time gains in the maintenance phase: indeed, anyone with any experience using TDD will confirm that code written using good TDD practices is significantly easier to understand and maintain.

Modern development practices such as ATDD and TDD are not easy: they require significant efforts in championing, training, mentoring, and managerial support. However there is little doubt that, when well applied, these techniques significantly improve code quality and reduce maintenance costs, but can also contribute to faster project delivery and thus reduced project costs.

John Ferguson Smart is an experienced consultant and trainer specialising in Agile Development Practices and Java-related technologies based in Sydney, Australia. An international speaker well known in the Java community for his many published articles and presentations, John helps organisations around the world to optimize their Java development practices, processes and infrastructures and provides training and mentoring in open source technologies, SDLC tools, and agile development. John is CEO of Wakaleo Consulting, a company that provides consulting, training and mentoring services in Enterprise Java and Agile Development. John is also the author of 'Java Power Tools' and of 'Jenkins: The Definitive Guide', and is currently working on a book on Behaviour Driven Development.

John also runs training courses in Agile Java Development Practices: sessions are coming up soon for Canberra, Sydney and Wellington, with other sites planned soon!

Comments

Hi and thanks for this great post. If you agree, I'll will ...

Hi and thanks for this great post. If you agree, I'll will post a french translation on my blog.

I've read the first mentionned study: I'm no reason to deny their conclusions - I'm fully convainced by ATDD & BDD - but I'm obliged to think QSM use it to promote his software solution.

And a small part is missing in first item of last bullet list in your post (time spen .... ?)

Regards