This article is about some changes in the ME Framework that were made in response to the feedback that we got from early adopters and are available in the third development release of ME Framework 1.2 that Mikhail just announced.

It is targeted to people who use (or plan to use) ME Framework or JT harness or work in the area of Java ME testing.

Foreword

Before the open source release, we were developing and using the ME Framework internally for years, as long as Java ME exists. As it turned out, to start using it outside of the internal SUN environment and supporting infrastructure is not a trivial task. Unlikely these problems are unique to our project.

Some things  became too natural for internal users. Some approaches that we use internally when develop TCKs with ME Framework help to optimize the work of the big production organization, responsible for development and sustaining of a big number of products. It turns out that people who need to do a first step, like to create a single test suite once, need more simple ways to do simple things.

Simplest Test Suite

The Simplest Test Suite example, that Alexander (aka Skavas) has recently checked into repository, is what its name means - simplest possible test suite, that we found reasonable for this product. It consists from three files - ant build script, test suite .jtt parameter file and a sample test, If you follow the instructions from the readme file, it will take 5 min to build and launch it plus time to download.

I would like to go into details of two issues, that were addressed in the ME Framework 1.2 3rd Development release and are demonstrated in this sample:

• Allowed alternative test formats, not limit to html test description, that have to be created for each test.
• Allowed not to provide explicit  information on what class and resource files are associated with test. This is necessary for automated packaging of test applications (bundles) and works now for simple tests.

Simple way to describe tests

The ME Framework is a plugin to JT harness (aka JavaTest ^TM harness), that enables Java ME test development and execution. This specific change was to correct usage of JT harness features by ME Framework, specifically, TestFinder.

TestFinder in JT harness is the class, responsible for providing array of TestDescription elements, that are representing test meta data in JT harness' model of the test suite. Support of different formats of tests is done in JT harness by plugging in corresponding extension of the TestFinder.

In short, the first change in ME Framework was just enabling TagTestFinder in the .jtt file. TagTestFinder uses javadoc tags to identify test and describe its meta information and it is a TestFinder, used in JTReg. HTMLTestFinder is more familiar to TCK users, it describes tests in specially formatted html table, check here for an example.

Simplest Test Suite uses tags to describe tests - it allows to decrease number of files, naturally supported by IDEs and somewhat familiar to people with JUnit experience. Check how the sample test looks like.

Other kinds of TestFinder

Other then Tag and Html finders, the standard set of TestFinder implementations, provided in JT harness, includes BinaryTestFinder, that reads information on tests to run from the binary file. It is the one that is most often used in TCKs, where there is a need to optimize time to read huge test tree. BinaryTestFinder is most applicable to cases, where test suite does not change after its development is done, because any change in any test description requires regeneration of the binary test data.. BinaryTestFinder is used in conjunction with another  TestFinder that reads test descriptions from tags or html files during the build of the test suite and stores them in the serialized form.

It's just sort of "extension" on top of other Finders, that"sits" on top of them, and gets the Descriptions from other Finders and serialize them to the binary data, and also allows to read the descriptions from the serialized data. It can be used with any other TestFinders that that do the actual job of parsing the TestDescriptions from HTML, javadocs, etc.

How to find what content to put into the .jar of the test application

This topic is more complex and hence more entertaining one.

The problem in more details. ME Framework allows for a great flexibility in how to turn huge pile of class and resource files of the test suite into set of MIDP application packages for iterative execution through autotest. Criteria that control creation of packages can be

• max size of the jar file
• number of tests to package into the single jar
• tests themselves may describe some conditions in their associated meta data that would regulate how these tests need to be packaged

Tests are packaged into .jar at runtime to allow the user to vary packaging parameters to achieve different goals, primarily to optimize network traffic and test execution time.

This all means - to form various application packages ME Framework needs to know which class and resource files are associated with each test.

These data can be calculated automatically using several methods, I will talk about them later. What we did for this sample: TestDescription in JT harness requires definition of the 'executeClass' parameter for every test. The value of this field is the name of the class that is test entry point, implementing one of supported interfaces.  If test consists from the single .java source file that defines single class - nothing to calculate, right  ? All class files of this test can be described by /$executeClass.*/ expression. Note that inner classes are OK here as well.

Other then this, nothing else happens in this sample automatically - if there are multiple .java files, that constitute a test, one need to list them manually in the 'resources' field.  Not that convenient as 'do nothing', but simple to explain :) Other then 'resources' field, the standard mechanism with special file named testClasses.lst still works, just generation of this file is not supported in the build of the Simplest Test Suite. The file testClasses.lst should be located in the special place in test suite's directory hierarchy and containing testURL-test classes pairs, check how it looks like in another sample.

Approaches to automatically calculate list of classes to package

Content of the testClasses.lst is calculated statically during the build of the test suite.

Approach 1 - individual test compilation.

One way to do this is to put the load on the compiler. Compile tests individually, set own target dir for each test, not give and test library classes on the classpath but provide reference to their source directory instead. All class files, that are needed by the test, will be in the target directory of the compiler. The described approach is simple and reliable one, we used it for some TCK releases. Weak places of it is that it is slow, at least in our implementation where java compiler and preverifier were executed through the command line for each test. 

Approach 2 - parsing class files for dependencies.

We do this with a simple tool based on JINI's ClassDep API.

It does static class analysis and should calculate all dependencies correctly except for cases like Class.forName is used in the tests (which is rarely needed). There are probably other tools which can do the same thing.

Here is the example of the command, that can be used in the test suite build script:

 > java -cp jini2_1\lib\tools.jar;%JAVA_HOME%\lib\tools.jar com.sun.jini.tool.ClassDep ^
    -cp <ALL TEST AND FW CLASSES> <executeClass> ^
    -out com.sun.tck.cldc -out com.sun.tck.j2me -out java -out sun -out javax

In cases where all necessary classes can not be found through parsing class files from the entry point, we used static, manually created table, that is merged into the generated testClasses.lst. This can be just a table with the same format, or, as mentioned above, just a 'resources' field of the TestDescription.

Limitations

Build script  for the sample will work only with simple automated tests for MIDP. Support of distributed, interactive, OTA or CDC tests will require more sophisticated build procedure, that will take care of compiling SE-side components, for example.

Yet more simplification

Further simplifying of this test suite and test development approach is possible by making it close to JTReg and xUnit. This can be:

• Minimizing set of mandatory meta data. For example, if a test consists from the single .java file, test entry point, that is now specified in the mandatory 'executeClass' parameter. is known.
• If this suite is used in the scenario, where tests are actively and constantly changing, it may be possible to eliminate 'build' stage, as it is done in JTReg,  implement test source compilation/preverification as a step in the test execution process. 

Acknowledgment

I would like to express my appreciation to the hosting provided by that allows to link sources for java.net projects from this blog a nice way that you can see above.