Rémi Forax's Blog

September 2006 Archives

Use foreach to iterate using an Iterator

When the foreach syntax (for(:)) has been introduced in 1.5, a recurring question was why foreach is not able to iterate using an iterator.

I think i have a trick to do that using the syntax of the closure :

 Iterator<String> scanner=new Scanner(System.in);
 for(String word:() { scanner })
   System.out.println(word);

Why this code works. The foreach statement needs an array or an object that implements java.lang.Iterable. Iterable is an interface that has a single method iterator that returns an iterator thus the closure convertion is applicable.

Rémi

Is function type required for closure ?

As seen in Neil Gafter's blogs, there are two rival proposals about adding closure to Java language. The first one, named v0.1, introduces a new kind of type, function type in order to express the type of a closure. The second one, named v0.2, enables to use interface as type of closure.
The goal of this entry is to show the pro and cons of the two proposals.

Let me take an example. I want to add each integer of a list, so i can use a closure for that. The code bellow uses the abbreviated invocation syntax of closure defined in section 10 of the two proposals.

 List<Integer> list=Arrays.asList(2,4,6);
 int result=0;
 forEach(int value:list) {
   result+=value;
 }
 System.out.println(result);

The code that declares and uses closure is the same for the two proposals. A contrario, the code that defines the method forEach is different depending of the version of the proposal.
With the proposal v0.2, closures are typed by an interface, like that:

 public interface Performer<E> {
   void doIt(E element);
 }
 public static <T> void forEach(
   Iterable<? extends T> iterable,Performer<? super T> performer) {

   for(T item:iterable)
     performer.doIt(item);
 }

The main interest of this way is the seamless integration with existing codes. The code of forEach is a "traditionnal" 1.5 code with no supplementary syntax. Futhermore, using an interface permits to name abstraction, it seems more clear to talk about a comparator than a int(T,T).

Now, the same example but with the syntax describes in the v0.1.

 public static <T> void forEach(
   Iterable<? extends T> iterable,void(T) func) {

   for(T item:iterable)
     func(item);
 }

Clearly, you have to learn a new syntax: void(T) is a function that take a T and returns nothing. Apart that fact, the syntax is not hard on the eyes like, by example, function pointer in C.

The subtyping rules defined between function types avoid to understand generics deeply, void(T) seems more simpler than Performer<? super T>.

In order to use closures with an existing code the v0.1 includes the same convertion mecanism between closure and interface that the v0.2. So you have two ways to define forEach, not a good point !

The bytecode used for invocation is different, the v0.1 will use invokedynamic and the v0.2 invokeinterface. The difference is that invokedynamic don't need to box and unbox, so the performance could be far better using a function type than using an interface. The cost of boxing could really be a problem because in general closure are small code and the cost of boxing will not be negligible comparing to the cost of executing the instructions of the closure. Perhaps, it's not a big beal, Java has lives with that cost since its inception and it doen't seem to be a big trouble.

And i want to end with a stupid argument, the proposal of the v0.1 takes less line of codes that the v0.2 :)

Now express yourself, what do you think ?

Synchronized or confined

A new version of the closure proposal has been post at the end of this week by Neal Gafter and proposes in section 3 to tag parameter with a special keyword to differenciate between synchonous and asychronous closure.

For me this section contains two flaws, the first one is that the keyword used is synchronized, the second one is to not recognize that such feature can be usefull in other contexts than closure.

In the Java API, there are existing methods that take an object as argument and don't allow to store this object in a field, by example, startElement/endElement of SAX handler or RowFilter.include().
I think a unique keyword can be used in all this cases.

Now, why not using synchronized for that ?
synchronized means thread, threading model, memory model, etc. This keyword is so important that it can't be reused to express something totally different without complexified something that is already not obvious.
I think that using an annotation in this case is better. You can name it as you want, there is already a reflexion support. I propose @Confined because the reference can't escape.

The following code shows the usage of @Confined with closure :

  public static <T> void forEach(Iterable<? extends T> iterable,@Confined void(T) func) {
    for(T item:iterable)
      func(item);
  }
  ...
  List<Integer> list=Arrays.asList(2,4,6);
  int result=0;
  forEach(list,(int value){
    result+=value;
  });
  System.out.println(result);

Rémi

With one breath, with one flow
You will know
Synchronicity
                             --- Synchronicity / Police

Using jrunscript as a build tool

In my last post, i've described how to use jrunscript to create a build script.
As olivier wrote in the comments, the build process is usually a dependency graph and jrunscript doesn't do that by default.
But javascript is far more powerful than XML thus it's easy to create commands that enable to declare dependencies betweeen functions as Ant or make does.

I propose the following commands:

1. depends_on that declares a set of dependencies of a function.
2. run_targets that explores the graph from a function in a dependency first order.

Javascript enables to put data in an existing object even a one like Object or Function. In fact, objects are considered as an hashtable of couples property/value so the dependencies of a function can be stored in the function itself.

The build script:

function test1() {
  echo("test 1");
}

function test2() {
  echo("test 2");
}
test2.depends_on(test1);

function test3() {
  echo("test 3");
}
test3.depends_on(test1,test2);

run_targets(test3);

And the script that creates the two commands :

function depends_on() {
  var dependencies=new Array();
  for (var i=0;i<arguments.length;i++) {
    dependencies.push(arguments[i]);
  }
  this.dependencies=dependencies;
}
Function.prototype.depends_on=depends_on;

function run_targets(target) {
  run_targets_internal(target,new Object())
}

function run_targets_internal(target,markers) {
  if (markers[target]=="X")
    return
    
  markers[target]="X"
  for each(dependency in target.dependencies) {
    run_targets_internal(dependency,markers);
  }
  
  echo("target "+target.name);
  target();
}

Now, it's time to drop Ant :)

Using jrunscript to create a build script

The JDK6 provides a new command jrunscript that enables to execute script shell in Java environment. By default, jrunscript uses javascript as scripting language and provides some useful default functions like cp, cd, cat, etc.

These global functions seem designed to ease the creation of build scripts, so i propose to show the basics of how to write such scripts.

First, the script must create the directory that will contain classes :

 // project properties
 srcDir='src'
 classDir='classes'

 // target init
 mkdirs(classDir);

To run the script, just type jrunscript yourScriptFileName.js in your shell
Then the script must compile the sources...
Weark, there is not function named javac that compiles sources to classes !!

But this function is easy to write using the package javax.tools introduced by the JSR199 and available in mustangjdk6.
In addition to javac, we need another function that gather all the source files of some directory, i've named it fileset in reference to the <fileset> of Ant.

So compiling a source tree is done with the following lines in javascript :

 function fileset(path,pattern) {
   var set=new Array()
   function callback(file) {
     set.push(file)
   }
   find(path,pattern,callback)
   return set 
 }

 // target compile
 echo('compile')
 javac(fileset(srcDir,'.*\.java'),classDir)

you can notice that callback is some kind of closure in javascript :)
The function javac is just a copy/paste of one example that you can find in the comment of the class javax.tools.JavaCompilerTool :

function javac(fileset,destDir) {
  compiler=javax.tools.ToolProvider.getSystemJavaCompiler()
  fileManager=compiler.getStandardFileManager(null,null,null)
  fileManager.setLocation(
    javax.tools.StandardLocation.CLASS_OUTPUT,
    java.util.Arrays.asList([new java.io.File(destDir)].valueOf()))

  compilationUnit=fileManager.getJavaFileObjectsFromFiles(
    java.util.Arrays.asList(fileset.valueOf()));
    task=compiler.getTask(null,fileManager,
                          null,null,null,compilationUnit)
  task.call()
  
  fileManager.close()
}

We now have a basic script that compiles a java project, ok, it doesn't have by example a way to declare dependency between target like Ant but that's a good starting point.
The whole script is available here and if you want to know more about scripting and Java, you can read the A. Sundararajan's Weblog.

Function that does not return normally II (the return)

In a previous entry, i've written about declaring a method that doesn't return normally using null, the type of null, as return type. A comment from Neal Gafter make me realize that i was wrong but i now think the closure spec is wrong too.

What the closure proposal says is that a function that doesn't return normally should use null. Not the converse, so i agree with neal that a method that use null as return type doesn't allow the compiler to flag the method as "never returned", perhaps the method always returns null.

But the closure proposal is wrong too, if a function that never returns is typed null its function type can't be a subtype of the type of a function that returns a primitive type.
So the example below will not compile :

 null(void) f={
   throw RuntimeException();
 };
 int(void) g=f;

The assumption of the closure proposal is that a function that never return has a return type that is a subtype of null. This is clearly wrong with the current Java type system because boxing relations are not subtyping relations.

I propose to introduce throws as the return type of a method that never returns, with the following subtyping rules :

 Object < Integer < null < throws
 and
 int < throws

Using throws as several avantages upon null for method that doesn't return normally because :

1. a function that returns throws is a subtype of a function that returns a primitive type.
2. users can declare methods that doesn't return normally and the compiler can understand that.

I wait your comments.

Closure and collection integration

The closure proposal doesn't define how closure and collections will work together. It's reasonable to say that this API will exist because closure in order to be accepted by the community must be well integrated with collections. After all, java.util is the second more used package after java.lang.
But that is not so easy.

In a dream world, collection should have method like forEach in their interface, but there is no way to add a method in an existing interface without breaking the sacredsanct backward compatibility.
So a code like that is not possible :

 List list=Arrays.asList("closure","collection");
 List list2=list.forEach(int(String s) {
   return s.length();
 });

In my opinon, there are two other ways to do something similar. The first one consists in reversing the problem, if you can't add a method on collection, you can add it to closure.
So closure can have a base class, say java.lang.Function, that defines methods that take collections as parameter.
So the privous code will become that :

 List list2=int(String s) {
   return s.length();
 }.forEach(list);

I'm not a big than of this solution, it's too PERL-like, you don't know what an instruction do without read the whole block.

The other solution is to defined methods that use closure in a class like java.util.Collections that already contains lot of helper methods and to use an import static.
In this case, the code will be :

 import static java.util.Collections.*;
 
 List list2=forEach(list,int(String s){
   return s.length();
 });

Or with the inline syntax of the closure proposal :

 import static java.util.Collections.*;
 
 List list2=forEach(list) int(String s) {
   return s.length();
 };

 package java.util;
 public class Collections {
   public <T,U> List<U> forEach(List<T> list,U(T) function) {
     ...
   }
 }

Any other ideas ?

JDK7: language enhancements

Wow, we begin to have a good view of the language enhancements planed for Dolphin JDK7.

1. XML Syntax : blog entry by Mark Reinhold
2. Super-packages : blog entry by Gilad Bracha
3. Closure : a blog entry by Peter Ahé
4. Annotations to detect software failure : JSR 305 submitted by Bill Pugh.

Like any geek, i'm not sure to be able to wait 2 more years before playing with these features.

Garden thought

I'm seating in my garden, my laptop on my knees. i'm hearing the noise of some mowers in the background, my kid is sleeping, my wife is trying to resolve a sudoku, and i'm thinking about closure, again !

Why doesn't enable to define a closure using a reference to a method ?

 public class HelloClosure {
   public void sayHello(String name) {
     System.out.println("hello "+name);
   }
   public static void main(String[] args) {
     HelloClosure hello=new HelloClosure();
     void(String) sayHello=hello.sayHello;
     sayHello("closure");
   }
 }

The syntax seems obvious, the code (hello.sayHello)("closure") is semantically equivalent to hello.sayHello("closure").
The compiler has only to verify that there is no field named identically in the scope before interprets it as a closure.

It's not very difficult to translate, the type of the closure is the type of the method, the referer of a non static method will be store in a field of the closure like any other local variable.

  public static void main(String[] args) {
     HelloClosure hello=new HelloClosure();
     void(String) sayHello=void(String s) {
       hello.sayHello(s);
     };
     sayHello("closure");
   }

 public class Application {
   public void onButtonAction(ActionEvent event) {
     System.out.println("damn, i'm clicked");
   }
   public static void main(String[] args) {
     Application application=new Application();
     JButton button=new JButton("Ok !");
     button.addActionListener(application.onButtonAction);
     ...
   }
 }

What do you think ?

This work is licensed under a Creative Commons License.