Skip to main content

Test Harness and Test Format

Posted by alexeyp on March 26, 2007 at 10:58 AM PDT

It is true that the most popular test format supported by
many Java TM IDEs is href="http://www.junit.org">JUnit and its
variations like TestNG
or J2MEUnit.
Interested to compare JavaTest
TM
harness and JUnit ?

JavaTest is a test harness, JUnit is a test format. JavaTest
is created to manage href="http://weblogs.java.net/blog/alexeyp/archive/2007/01/test_harness_fo.html">test
suites written in many different formats, JUnit is a specific
format and API for test development. The comparison is invalid, it would be more appropriate to compare JUnit API with one of test APIs, included into JavaTest's core libraries or its extensions.

Test format that is traditionally associated with this
harness is the
one that is used in most of Java Technology Compatibility
Kits (TCK),
based on test meta data provided in html form and Test/MultiTest API.
Check the href="http://weblogs.java.net/blog/alexeyp/archive/2007/02/testing_command_1.html">previous
article for example of how JavaTest's Test Description looks
like in .html.
Comparing to JUnit, the java code of the TCK test may look like this:

cellpadding="2" cellspacing="2">
public void testadd() style="color: rgb(0, 153, 0);">throws
AssertionFailedException
{

    System.out.println("add");

    math.Arithmetic instance =
Arithmetic.getInstance();

    for (int a=0;a<=10;a++)

    for (int b=0;b<=10;b++) {

       
int expectedResult = a+b;

       
int result = instance.add(a,b);

        style="color: rgb(51, 204, 0);">assertEquals(expectedResult,
result);

    }

}
public Status
testadd() {

    log.println("add");

    math.Arithmetic instance =
Arithmetic.getInstance();

    for (int a=0;a<=10;a++)

    for (int b=0;b<=10;b++) {

       
int expectedResult = a+b;

       
int result = instance.add(a,b);

        style="color: rgb(51, 204, 0);">if (expectedResult !=
result)

style="color: rgb(51, 204, 0);">           
return Status.failed( style="color: rgb(51, 204, 0);">"Expected: "+ style="color: rgb(51, 204, 0);">expectedResult style="color: rgb(51, 204, 0);">+

           
      style="color: rgb(51, 204, 0);">" Received: "+ result style="color: rgb(51, 204, 0);"> style="color: rgb(51, 204, 0);">);

    }

   
return Status.passed("OK");


}

The choice of the test API with explicit return for both
positive and negative test conditions was driven by its use in TCKs,
test suites, that are used for certification. One reason is requirement
to be able to provide maximum to evaluate test failures offline without
access to the implementation, where test failure is reproduced. Because
of this there 'log' API is used instead of 'System.out', there is a
special class Status that holds additional information other then
pass/fail indication. Another reason is that TCK tests verify specific
assertions of the specification and action of qualifying test as Passed
need to be concise.

The JTReg
is
a JavaTest with set of
plugins tuned for JDK regression testing. As opposite to the primary
JavaTest use as a test harness for certification test suites, like TCK,
where
main requirements come from the test suite end user, JTReg is targeted
to JDK developers. Main goals here are easiness to write and execute a
test, support for major test types, possibility to execute shell
scripts, adoption to the JDK development process. As you can href="https://openjdk.dev.java.net/jtreg/faq.html#question2.1">see,
there API for JDK regression tests is even simpler. See also JTReg FAQ
for
comments
about JUnit support
in JavaTest and JTReg.

JDTS
Framework is a set of JavaTest harness plugins,
that provide support for their own format, that is specially focused at
functional and quality testing for Java ME.

Overall, different needs and different uses result in
different test formats, that can be supported with single test harness.

Related Topics >>