Rémi Forax's Blog

Rémi Forax is Maitre de Conférence at University of Marne-la-Vallée since 2003 where he obtained his PhD on multi-method implemntation in Java. He has been using Java for many years and enjoys himself hacking the JDK.

Alternative syntax for BGGA closure

In my last blog entry, i've said it was time to discuss about the BGGA closure syntax. So here is my proposed syntax.

The closure syntax

There are two parts in a closure syntax, the first one defines the closure type, the second one defines the expression (not a statement) of the closure. By example, with a BGGA closure { int x => x+2 }, the expression part is x+2. BGGA uses a trick for closure type, it infers the return type using the result of the expression so the closure type is a function that takes an integer and returns an integer.

Why BGGA syntax is not great

In my opinion for multiple reasons:

1. Closure is delimited by curly braces, but in Java curly braces is used for defining blocks and not expressions.
2. In Java unlike Scala function return type is defined before the parameter types but BGGA function type define return type after parameter types {char,char => boolean}.
3. As already said => is too close from <= (less than equals).
4. Mixing parenthesis and curly braces is hard to read, anonymous class already do that and it's a pain point for beginners (and IDE :)
   new Thread({=> System.println(); }).start()
5. Curly braces are already used as expression to initialize arrays or annotation values.
6. Function type syntax {int,int=>int} is too close from closure one {int x,int y=>x+2}; but these things are different concepts.
7. { int => int } and { int ==> int } means to different things but syntaxes are too similar and i'm not able (yes really) to recall wich one defines restricted closure.
8. BGGA closures are defined as instructions followed by an expression, so you can write

            { String s, int n => String r = ""; for ( ; n > 0; n--) { r += s; } r };
          

   which is equivalent to this code

     static String concat(String s, int n) {
       String r = "";
       for ( ; n > 0; n--) {
         r += s;
       }
       return r;
     }
     ...
     #concat(String,int)
          

   Guess which one is more readable. In my opinion, closure should be restricted to an expression.

My proposed syntax

  expression closure = closure_parameters  expr
  closure_parameters = '|' parameters lists '|'

Examples:

   RemisClosure = |int a, int b| a + b;
   
   Integer[] primes = { 19, 23, 2, 11, 17, 31, 5, 13 };
   Arrays.sort(primes, |Integer x, Integer y| y.compareTo(x));

The syntax is close to Ruby's one, as I previously said, closures are expressions because in Java, x+y is an expression, no need to add parenthesis, brackets or curly braces. Parameter types are enclosed by pipe ('|') to avoid confusion with other braces.

Closure in other languages:

  BGGAClosure       = {int a, int b => a + b };
  GroovyLikeClosure = {int x, int b -> a + b };
  RubyLikeClosure   = { |int a, int b| a + b };
  FanClosure       := |int a, int b -> int| { return a + b }

Gotcha: closure with no parameter type

In Java void is not a parameter type, so |void| 2+3 is not a valid syntax for a closure that takes no argument and return an int.
I see two syntaxes:

1. || 2+3, even if there is not clash with boolean or, it could confuse beginner.
2. 2+3 this means there is an automatic conversion from an expression to closure. It adds another pass to the overload resolution algorithm, which is already complex because it handles boxing, varags and generics.

Function type

In Java, return type is declared before parameter type, I don't see why function type should break that rule. So a function type is a return type followed by parameter types separated by comma and enclosed in parenthesis. This syntax is very similar to FCM one.

Update ask by Neal, grammar of function type:

  return_type '(' parameter_type_list? ')'

Example:

  f(2, |int x| x+2);
  ...
  static int f(int value, int(int) closure) {
    return closure.invoke(value);
  }
  
  g(|int x| |int y| x+y);
  ...
  static int g(int (int ()) closure) {
    return closure.invoke(2).invoke(3);
  }

  int (char[]) throws IOException read =
    reader#read(char[]);

  int lengthSum=sum(|String s| s.length(), "toto", "tutu");
  ...
  static <V> int sum(int (V) f, V... values) {
    int sum=0;
    for(V value:values) {
      sum+=f.invoke(value);
    }
    return sum;
  }

Closure block

A closure block is a block of instructions that can be passed to a method in order to perform control abstraction.
A closure block is declared like this:

  expr = method_call closure_parameters? block_body

Example:

   Reader reader=...
   with(reader) {
     ...
   }
   ...
   public static void with(Closeable closeable) void f() {
     try {
       f.invoke();
     } finally {
       reader.close();
     }
   }

This syntax declares the closure block type after the method parameters (like in Scala). The syntax has to be different from a method that takes a function type as parameter because

• closure block let you use return, break, continue.
• a method that takes a function block performs exception transparency transparently.
• a closure block type is not a real type, it cannot appear somewhere else.

  method = method_decl closure_block_decl* method_body

  closure_block_decl = name '(' parameters ')'

Closure block return type is always 'void', so the syntax ommit it. The variable containing the closure block is not useable as left value.
Examples

  static void forEach(int... nums) block(int) {
    for (int n: nums) {
      block.invoke(n);
    }
  }
  ...
  int sum = 0;
  forEach(2,3,4) |int n| {
    sum += n;
  }
  ...
  static boolean hasNegativeValue(int... nums) {
    forEach(nums) |int n| {
      if (n<0)
        return true;
    }
    return false;
  }

I don't think a specific syntax for loop abstraction worth the need to learn a new syntax.

 void forEachEntry(Map<K,V> map) block(K,V) {
    for(Map.Entry<K,V> entry : map.entrySet()) {
        block.invoke(entry.getKey(), entry.getValue());
    }
}
...
Map<String, String> map =
forEachEntry(map.entrySet()) |String first, String second| {
  ...
}

Conversion between function type and block type

It's possible to pass a function type as argument of method that declares a closure block type. The opposite is not true because closure block can do non local transfer but you can use a cast.

  void invokeInEDT() closure() {
    void() f=closure;          // illegal conversion
    final void() f=((void())closure; // ok

    EventQueue.invokeAndWait(new Runnable() {
      public void run() {
        f.invoke();
      }
    });
  }

This entry is already too long, so i stop here. I wait your comments.
Rémi

Source of BGGA prototype available

After being frozen more than 3 months, the BGGA closure dev is resumed, the sources of the BGGA prototype have been pushed 3 hours ago by Neal Gafter.
The sources are available in the closure workspace: http://hg.openjdk.java.net/closures/closures/langtools/ .

Troll: I think it's time to reconsider the syntax to be less alien or from mars.

Parameterized type are NOT inherently unsafe

Time to time, i heard that sentence "array of generics a inherently unsafe" or a variation.
That is not true and i would like to explain why and even better propose a way to improve the user experience.

Array of generics ?

First, there are two kinds of "generics", type variable and parameterized type, and they behave differently.

An example of array of type variable:

  class A<T> { // T is a type variable
    T[] m() {        // T[] is an array of type variable
      ... 
    }
  }
  ...
  A<String> a =new A<String>();

Here, T is a type variable. A>String> is a parameterized type, it's an instantiation of A<T> with T = String.

As Gilad Bracha in the generics tutorial wrote "The component type of an array object may not be a type variable or a parameterized type, unless it is an (unbounded) wildcard type",
currently the specification doesn't allow to create an array of type varaible and array of parameterized type.

Why ?

Let's take the example given by Gilad:

  List<String>[] lsa = new List<String>[10]; // not really allowed
  Object o = lsa;
  Object[] oa = (Object[]) o;
  List<Integer> li = new ArrayList<Integer>();
  li.add(new Integer(3));
  oa[1] = li;                     // problem here
  String s = lsa[1].get(0);

Generics in Java are not reified, i.e. at runtime there is no difference between a List<String> and a List<Integer>, they are all List. So the VM can perform runtime check to distinguish between the two parameterized type.
Just before the bytecode generation, in a pass named erasure, the compiler transform all parameterized types to types without type arguments and replace type variables by their first bound.

So our example is erased to:

  List[] lsa = new List[10]; // not really allowed
  Object o = lsa;
  Object[] oa = (Object[]) o;
  List li = new ArrayList();
  li.add(new Integer(3));
  oa[1] = li; // problem but not detected here
  String s = lsa[1].get(0); // run-time error - ClassCastException

So when VM execute the line oa[1] = li; it doesn't detect a problem. The VM generate a ClassCastException later here at the next line.

To avoid to have a code that can generate a CCE without any warning, the JSR15 experts decide that array of parameterized type can not be created.

Here comes the pain

Because you often need to create an array of parametrized type, you have to use painful workaround.
In a recent interview about Effective Java, Joshua Bloch talk about "The "impedance mismatch" between arrays and generics can be a pain."

An example of such tricks, the creation of an array of parametrized type is replaced by the creation of an array of wildcard then a cast.

  List<String>[] list =
      (List<String>[])new ArrayList<?>[1]; // warning

These workaround codes are very dangerous because if one day in the future generics are reified (JDK8 :), these codes will not work anymore and throw a ClassCastException.

So what ?

So array creation is prohibited because it can lead to unsafe code. I think it's time to say:
Sorry we goof, we can do better.
Instead of preventing to create array of parametrized type we can just generate a warning where needed.
What is the problem in our running example ? The line oa[1] = li; is unsafe because a List of Integer is store in an Object.

In general terms, the problem is that when converting an array of parameterized type to a type without type argument, the type is lost (by definition) so neither the compiler nor the VM (because of the erasure) can garantee the type safety.

Unsafe convertion

Using a JLS like wording, let T and U such as U is the declared type of u and T is defined by, T t=u;
If U is an array of parametrized type, the compiler may emit a warning, if :

• T is Object, Cloneable, Serializable or
• T is an array of V which is not parametrized or
• T is an array of raw type or
• T is an array of unbunded wilcard.

If this rule is added, creation of array of parameterized type is safe and then can be allowed.

And what about the backward compatibility ?

With this new rule, program that compile will continue to compile. Workarounds are not needed any more. But some code that currently don't raise a warning can now raise a warning.
There are in my opinion two different cases:

  SelectionKey key = ...
  List<String> list = ...
  
  key.attach(list); // will generate a new warning

It's not a big deal because the following code already generate a warning.

 List<String> list = 
    (List<String>)key.attachment(); // already generate a warning

The second case is more problematic

  List<String> list = ...
  System.out.println(list);   // warning

Here, println take an Object, so the parameterized list is assigned to an Object. Hum, such warning is less desirable.
Here, the compiler doesn't know that the only things done to that object is to call toString() on it so it's safe.
To avoid such warning, i propose to add a new key to @SuppressWarnings, @SuppressWarnings("parameterized-type") in order to indicate that assign/convert a parameterized type is safe in that case.

  public void println(@SuppressWarnings("parameterized-type") Object x) {

I wish and hope, these changes can be included in JDK7.

Is array of type variable inherently unsafe ?

Yes, array of type variable are inherently unsafe. See the following code.

  class A<T> {
    T[] m() {
      return new T[1]; // erasure create a, array of Object 
    }
  }
  ...
  A<String> a = new A<String>();
  String[] s = a.m(); // oups

Javac + invokedynamic

Just for fun, this morning, i've patched the java compiler to be able to generate classes that use invokedynamic instead of invokevirtual/invokeinterface when invoking a method. following the JSR292 EDR

The patch is based on the source of the langtools repository of the hotspot project, so to apply the patch, first clone the repository

  hg clone http://hg.openjdk.java.net/jdk7/hotspot/langtools/
 

import java.dyn.InvokeDynamic;

public class Test {
  @InvokeDynamic
  public void m(String message) {
    System.out.println(message);
  }
  
  public static void main(String[] args) {
    // here m is invoked dynamically
    new Test().m("hello invoke dynamic");
  }
}

And now, how to run it.
Hum, good question, i'm afraid we have to wait the RI implementation of JSR 292 :)
Or perhaps an enhanced version of the IKVM Proof of Concept.

Cheers,
Rémi

July 2008

Search this blog:

Categories

Archives

July 2008
May 2008
April 2008
March 2008
February 2008
January 2008
November 2007
October 2007
September 2007
August 2007
June 2007
May 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006

Recent Entries

Alternative syntax for BGGA closure

Source of BGGA prototype available

Parameterized type are NOT inherently unsafe

This work is licensed under a Creative Commons License.