Cay Horstmann's Blog

The World's Simplest Unit Testing Framework

I teach computer science at San Jose State University. In my experience, the habits from the first programming course stay with students for a long time and are very difficult to break. (I am always amazed how many seniors in my software engineering course still use Notepad to edit their Java files.) I want to get the freshmen to embrace test-driven development right from the get-go.

JUnit 4 is nice and simple. Just add @Test, and use assert, which is a standard Java feature. But unfortunately, that's still too hard for beginners. Did you every try to get a bunch of freshmen to add a JAR to their class path? It's not a pretty sight.

So, this is what I ended up doing. I require that every program prints out the value that the student expects:

public class BankAccountTester
{
   public static void main(String[] args)
   {
      BankAccount momsSavings = new BankAccount(1000);
      momsSavings.deposit(100);
      double balance = momsSavings.getBalance();
      System.out.println("Balance: " + balance);
      System.out.println("Expected: 1100");
   }
}

The printout is

Balance: 1100
Expected: 1100

That's the “green bar” moment in the world's simplest unit test framework.

I wrote an Ant extension that automatically checks the output and deals with silly issues such as roundoff:

Balance: 1034.2999999997
Expected: 1034.30

The students don't run Ant; that's just for my grading automation. My grading script contains an entry

    <java classpath="${submit.dir}"
          classname="${mainclass}"
          failonerror="true"
          outputproperty="mainclass.out"
          fork="true" />
    <echo message="${mainclass.out}" />
    <condition property="tester.fail" value="Output not as expected">
      <not>
        <asexpected value="${mainclass.out}" tolerance="${test.tolerance}" />
      </not>
    </condition>

There you have it. TWSUTF is System.out.println("Expected: ...") and an Ant condition that checks the output.

Big deal, right? Well, it turns out to be a bigger deal than I thought. The next assignment asks students to write a class IceCreamCone. And I cruelly ask them to write the test first.

public class IceCreamConeTester
{
   public static void main(String[] args)
   {
      double height = 6;
      double diameter = 1;
      IceCreamCone waffleCone = new IceCreamCone(height, diameter);
      double volume = waffleCone.getVolume();
      System.out.println("Volume: " + volume);
      System.out.println("Expected: How the !@#$ am I supposed to know?");
   }
}

Now I have them where I want them. They have to figure out the answer by hand for at least one case. That's very different from running the program and saying “ok, looks about right”.

Of course, after a few weeks, the simple tests get tedious. We want multiple tests without writing lots of static methods—another no-no in the intro course. And we want the other tests to keep on trucking when one of them throws an exception. That's the time to break out JUnit. Now it solves a problem that the students have.

What do you think? Should students get immersed in TDD in their first Java course, or would it be a better use of time to learn another sorting algorithm, or some GUI programming, or whatever?

Comments

• Please, oh, please, continue teaching TDD! It is incredibly hard to get a "solidified" programmer to even start listening that they may consider using TDD!

  Posted by: tobega on October 25, 2006 at 08:05 AM

• I also teach Java at a college. I am now trying to teach TDD too. This is very hard for students but I'm going to keep doing it. I like your very simple approach, I'm making them write test classes. My class is not their very first programming experience though, so I want more from them. I think you should keep it up!

  Posted by: eknapp on October 25, 2006 at 08:18 AM

• Sounds like NIH syndrome here and "For every problem there is one solution which is simple, neat and wrong.". This is one of those solutions.

  It is dead easy to setup an Ant script containing JUnit/compile/coverage targets + directory structure + necessary jars and zip it for your students. Then they can drop their JUnit tests in place and learn how things are done in the real world.

  Besides, how do you measure code coverage, generate test reports, create testSuites etc. etc. with your approach?

  Posted by: applebanana8 on October 25, 2006 at 11:11 PM

• 
  Interesting. I'm hitting some of the same issues in graduate courses. Classpath isn't any easier there either. I'm actually working on an article on nothing but setting up the classpath and sourcepath.

  When it comes to editors, the first assignment I give is to take a screen shot of jEdit showing Hello World. This assures that

  They learn how to take a screen shot.
  They have downloaded, installed, and run at least one decent editor.
  They learn that it is possible to write real applications in Java.

  I agree that JUnit is too much for an intro course. In my course I actually have students write their own very simple test framework. JUnit hides too much. I want them to see how the tests work.

  There is one problem with your test framework though. It doesn't scale because it requires manual inspection of the results. An important principle is that passing tests shut up. Output should be generated only if the tests fail. Otherwise, it's too easy to miss a failed test in a sea of passing tests. If you like, a test suite can print "All tests passed" just so you know it did run; but nothing else should be generated unless there's a failure.

  Posted by: elharo on October 26, 2006 at 05:20 AM

• I teach TDD to graduate students using William Blake's Test First Challenge
  http://www.xp123.com/xplor/xp0201/index.shtml
  since three years and it works really well.
  Add an ant script, Findbugs, PMD and Checkstyle and you increase in six weeks a lot the average programming skills of the students.

  Just my two cents.

  Posted by: leberre on October 26, 2006 at 06:40 AM

• I've been teaching test-first programming to seniors and grad students (see http://www.testingeducation.org/articles/ExperiencesTeachingTDD.pdf ) and starting last term, to first eyar students.

  We use JUnit, as integrated into the Eclipse programming environment. As noted by others, students who already know how to program are often quite resistant to programming test-first.
  In the first-year course, I use Langr's Agile Java and Savitch's Java: An Introduction to Computer Science & Programming. Langr is an introduction to test-first programming in Java but is much more appropriate to someone who already knows how to program than to a new programmer. After using it twice with first-year students, I will probably use my own course notes (or someone else's text if one appears on the market) rather than trying Langr again. Savitch is written much more appropriately (in tone, style, assumptions about what is new, etc.) than Langr, but it is a test-last approach that encourages many students to write in a more traditional (rather than O-O) style.
  My experience with first-year students is that it takes them a lot of work to get to test-first because they have to learn about classes, objects, and layered architectures very early in their programming experience. (In my class, we move to test-first as soon as we get past Hello World.

  I find that two techniques are very helpful for getting students past this challenging cognitive hurdle.

  First, I've started creating videos that illustrate the approach step by step. Here are links to a set that answer the first problem set in Langr's book:
  http://www.testingeducation.org/CSE1001/videos/Lesson1Part1.wmv
  http://www.testingeducation.org/CSE1001/videos/Lesson1Part12wmv
  http://www.testingeducation.org/CSE1001/videos/Lesson1Part13wmv
  http://www.testingeducation.org/CSE1001/videos/Lesson1Part14wmv
  http://www.testingeducation.org/CSE1001/videos/Lesson1Part15wmv
  http://www.testingeducation.org/CSE1001/videos/Lesson1Part6.wmv

  These are Creative Commons licensed. Feel free to reuse them.

  If we could agree on (or create) a good test-first introduction to programming for new programmers, we could create a pool of videos like this.

  If you would like to organize such a project (and want me to participate) or want to help me organize such a project, let me know. kaner@kaner.com

  Second, I have reluctantly concluded that it is important to pair with every student. It makes a big difference when I sit with a student and solve a problem with them. The way this works is that they do all the typing, but I ask leading questions when they get stuck and answer questions about the conceptual approach whenever they have them. I also suggest resources they can check (the java api, google with some suggested search terms) when they have a question about java syntax or an algorithm, etc. We work through an assignment, test-first, start to finish. This takes 1-3 hours per student and it seems to make such a noticeable difference that I think I should be doing this with each student in the first 6 weeks of class. (The bigger the class size, the harder this is. But "hard to achieve" is different from "important to do.") With so many new concepts to master at the same time as they are exposed to syntax, a compiler and an unforgiving machine, most students need coaching or they will fail.

  Posted by: cemkaner on October 27, 2006 at 12:09 AM

• Completely agree with applebanana8. How are you going to change these guys from using System.out.println 'testing' in the future? If you provide the aforementioned zipped project structure then they don't even have to care about the classpath. Actually they shouldn't have to, nor should we have to when we touch projects created by other people.

  When I create a new project I create the project directory and the build scripts from a template. It's always the same. You put (or symlink) the needed libraries in the lib directory and ant picks them from there. There are two scripts (a cmd and an sh for windows and u*x) to run the aplication that also take the libs automagically from the lib directory. The classpath is assembled by the scripts, or when running the test by ant, so nobody ever have to touch touch their classpath.

  Yes, of course it sounds a bit more complicated than doing System.out.println's, but that's how it should work. And actually using it isn't any more harder than the println version.

  Posted by: atleta on December 15, 2006 at 05:19 AM

• Here is a nice write-up about Unit Testing, especially nice for beginners.

  Unit Testing 101

  Posted by: screenedtwenty on May 18, 2007 at 05:28 PM

This work is licensed under a Creative Commons License.