Rémi Forax's Blog

Community: JDK Archives

Parameterized type are NOT inherently unsafe

Time to time, i heard that sentence "array of generics a inherently unsafe" or a variation.
That is not true and i would like to explain why and even better propose a way to improve the user experience.

Array of generics ?

First, there are two kinds of "generics", type variable and parameterized type, and they behave differently.

An example of array of type variable:

  class A<T> { // T is a type variable
    T[] m() {        // T[] is an array of type variable
      ... 
    }
  }
  ...
  A<String> a =new A<String>();

Here, T is a type variable. A>String> is a parameterized type, it's an instantiation of A<T> with T = String.

As Gilad Bracha in the generics tutorial wrote "The component type of an array object may not be a type variable or a parameterized type, unless it is an (unbounded) wildcard type",
currently the specification doesn't allow to create an array of type varaible and array of parameterized type.

Why ?

Let's take the example given by Gilad:

  List<String>[] lsa = new List<String>[10]; // not really allowed
  Object o = lsa;
  Object[] oa = (Object[]) o;
  List<Integer> li = new ArrayList<Integer>();
  li.add(new Integer(3));
  oa[1] = li;                     // problem here
  String s = lsa[1].get(0);

Generics in Java are not reified, i.e. at runtime there is no difference between a List<String> and a List<Integer>, they are all List. So the VM can perform runtime check to distinguish between the two parameterized type.
Just before the bytecode generation, in a pass named erasure, the compiler transform all parameterized types to types without type arguments and replace type variables by their first bound.

So our example is erased to:

  List[] lsa = new List[10]; // not really allowed
  Object o = lsa;
  Object[] oa = (Object[]) o;
  List li = new ArrayList();
  li.add(new Integer(3));
  oa[1] = li; // problem but not detected here
  String s = lsa[1].get(0); // run-time error - ClassCastException

So when VM execute the line oa[1] = li; it doesn't detect a problem. The VM generate a ClassCastException later here at the next line.

To avoid to have a code that can generate a CCE without any warning, the JSR15 experts decide that array of parameterized type can not be created.

Here comes the pain

Because you often need to create an array of parametrized type, you have to use painful workaround.
In a recent interview about Effective Java, Joshua Bloch talk about "The "impedance mismatch" between arrays and generics can be a pain."

An example of such tricks, the creation of an array of parametrized type is replaced by the creation of an array of wildcard then a cast.

  List<String>[] list =
      (List<String>[])new ArrayList<?>[1]; // warning

These workaround codes are very dangerous because if one day in the future generics are reified (JDK8 :), these codes will not work anymore and throw a ClassCastException.

So what ?

So array creation is prohibited because it can lead to unsafe code. I think it's time to say:
Sorry we goof, we can do better.
Instead of preventing to create array of parametrized type we can just generate a warning where needed.
What is the problem in our running example ? The line oa[1] = li; is unsafe because a List of Integer is store in an Object.

In general terms, the problem is that when converting an array of parameterized type to a type without type argument, the type is lost (by definition) so neither the compiler nor the VM (because of the erasure) can garantee the type safety.

Unsafe convertion

Using a JLS like wording, let T and U such as U is the declared type of u and T is defined by, T t=u;
If U is an array of parametrized type, the compiler may emit a warning, if :

• T is Object, Cloneable, Serializable or
• T is an array of V which is not parametrized or
• T is an array of raw type or
• T is an array of unbunded wilcard.

If this rule is added, creation of array of parameterized type is safe and then can be allowed.

And what about the backward compatibility ?

With this new rule, program that compile will continue to compile. Workarounds are not needed any more. But some code that currently don't raise a warning can now raise a warning.
There are in my opinion two different cases:

  SelectionKey key = ...
  List<String> list = ...
  
  key.attach(list); // will generate a new warning

It's not a big deal because the following code already generate a warning.

 List<String> list = 
    (List<String>)key.attachment(); // already generate a warning

The second case is more problematic

  List<String> list = ...
  System.out.println(list);   // warning

Here, println take an Object, so the parameterized list is assigned to an Object. Hum, such warning is less desirable.
Here, the compiler doesn't know that the only things done to that object is to call toString() on it so it's safe.
To avoid such warning, i propose to add a new key to @SuppressWarnings, @SuppressWarnings("parameterized-type") in order to indicate that assign/convert a parameterized type is safe in that case.

  public void println(@SuppressWarnings("parameterized-type") Object x) {

I wish and hope, these changes can be included in JDK7.

Is array of type variable inherently unsafe ?

Yes, array of type variable are inherently unsafe. See the following code.

  class A<T> {
    T[] m() {
      return new T[1]; // erasure create a, array of Object 
    }
  }
  ...
  A<String> a = new A<String>();
  String[] s = a.m(); // oups

Javac + invokedynamic

Just for fun, this morning, i've patched the java compiler to be able to generate classes that use invokedynamic instead of invokevirtual/invokeinterface when invoking a method. following the JSR292 EDR

The patch is based on the source of the langtools repository of the hotspot project, so to apply the patch, first clone the repository

  hg clone http://hg.openjdk.java.net/jdk7/hotspot/langtools/
 

import java.dyn.InvokeDynamic;

public class Test {
  @InvokeDynamic
  public void m(String message) {
    System.out.println(message);
  }
  
  public static void main(String[] args) {
    // here m is invoked dynamically
    new Test().m("hello invoke dynamic");
  }
}

And now, how to run it.
Hum, good question, i'm afraid we have to wait the RI implementation of JSR 292 :)
Or perhaps an enhanced version of the IKVM Proof of Concept.

Cheers,
Rémi

Closure and groovy builder

One think i really like in Groovy, it's its concept of Builder.
It allows to simply create tree of objects like XML trees using a concise syntax.

An HTML tree in Groovy is defined like that

  html {
      head {
        title "hello groovy builder"
      }
  }

JavaFX uses a quite similar syntax but the syntax is built-in. The beauty of the groovy beast is that the builder syntax relies on closure.

Closure

The trick is the following:

1. An XML node is a method.
2. The content of a XML Node is a closure taken as argument of the method

Ok, so let's try to do the same in Java using the BGGA closure proposal. CICE is too verbose for that case and it should work seamlessly with FCM+ JCA

Proxiiiiiiiiii

But before, how to define all the allowed markups of an XML dialect ?
Ok, we need one method by markup, but because the code is identical for two different markups, instead of writing boilerplate code, I will use the class java.lang.reflect.Proxy.
To use the Proxy, we have to define an interface. The one below allow us to generate XHTML.

public interface XHTMLBuilder {
  public void html(Object textOrClosure);
  public void head(Object textOrClosure);
  public void title(String text);
  public void body(Object textOrClosure);
  public void h1(Object textOrClosure);
  public void br();
}

A Proxy acts as a multiplexer a call to any of its methods is redirected to a single generic method named invoke.

public class XMLBuilderFactory {
  final Appendable appendable;
  
  public XMLBuilderFactory(Appendable appendable) {
    this.appendable=appendable;
  }
  
  public void text(CharSequence seq) {
    try {
      appendable.append(seq);
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
  }
  
  public <T> T createBuilder(Class<T> generatorInterface) {
    return generatorInterface.cast(Proxy.newProxyInstance(
      generatorInterface.getClassLoader(),
      new Class<?>[]{generatorInterface},
      new InvocationHandler() {
        public Object invoke(Object proxy, Method method, Object[] args)
            throws Throwable {
          
          ...
          return null;
        }
      }));
  }
}

Here appendable acts as a writable stream of characters, something on which I can append characters.
The method createBuilder takes an interface as argument and returns a class that implements that interface. All methods of thi interface are routed to the method invoke of the InvocationHandler.

Java Builder

Now, suppose that the method invoke is written, we will see after how to write it.
To write an XHTML tree on the standard ouput, using the BGGA closure syntax, the code is:

public class Main {
  public static void main(String[] args) {
    XMLBuilderFactory factory=new XMLBuilderFactory(System.out);
    XHTMLBuilder b=factory.createBuilder(XHTMLBuilder.class);
    
    b.html({=>
      b.head({=>
        b.title("hello Java 7 builder");
      });
      
      b.body({=>
        b.h1("Hello");
        factory.text("Greetings from");
        b.br();
        factory.text("Java 7 builder");
      });
    });
  }
}

It's even more readable if we use the control invocation syntax.

    b.html() {
      b.head() {
        b.title("hello Java 7 builder");
      }
      
      b.body() {
        b.h1("Hello");
        factory.text("Greetings from");
        b.br();
        factory.text("Java 7 builder");
      }
    }

This construct has several advantages. I can't produce ill formed XML, markup are opened and closed by the same code. Because the builder uses an interface I can't create markup with the wrong name. Even better, refactoring the markup name will works !

How to write the method invoke ?

For each argument of the method, i have to determine if it's a closure or any other objects. If it's a closure, i have to invoke it. If it's another object, i can append it into the Appendable.
The question is how to dynamically test if an object is or not a closure.
I don't want any closure i just want a closure that takes no parameter and returns void, a {=>void}.

  if (arg instanceof {=>void}) {
    (({=>void})arg).invoke();
  }

There is two poblem with that code. First, there are two kinds of closure (function type), unrestricted and restricted one. A restricted closure is a closure that can't break the control flow using break, continue or return. I am not sure i want my closure be able to break the control flow but the control invocation syntax creates unrestricted closure, so i need unrestricted closure. So instead of using '=>' in the function type, i have to use '==>'
By the way, i think the spec should be revised to swap the two notations ('=>' and '==>'), '=>' should be used for unrestricted closure. If i want a restricted closure which is not the default case, i have to take care about that and use the '==>' syntax. Append a second '=' is equivalent to say don't worry compiler i know what i'am doing.
So currently, the code should be:

  if (arg instanceof {==>void}) {
    (({==>void})arg).invoke();
  }

But there is another problem, currently, {==>void} is a parameterized type and so it's illegal to use it in an instanceof. Here, i have ask Neal, he says he planed to make "types such as this one non-generic".
There is a workaround, currently the closure prototype works with 1.4/1.5 VM and thus generates interfaces corresponding to function types. The interface corresponding to the raw type of {==>void} is javax.lang.function.unrestricted.V.
Here is the code that works:

  String name=method.getName();
  if (args==null || args.length==0) {
    appendable.append('<'+name+"/>");
    return null;
  }
          
  appendable.append('<'+name+'>');
  try {
    for(Object arg:args) {
      if (arg instanceof javax.lang.function.unrestricted.V) {
        ((javax.lang.function.unrestricted.V)arg).invoke();
      } else {
        appendable.append(arg.toString());
      }
    }
  } finally {
    appendable.append("');
  }

A zip containing the wole codes is here: closure-builder.zip
You need the prototype of the BGGA closure too.

Cheers,
Rémi

Da Vinci runtime properties

After a week without any internet access point surfing the snow of the Alp, monday, my fingers was eager to touch the keyboard again. Why not finishing my prototype of runtime properties that use the Da Vinci VM (i really love that name).

One ugly thing of the draft v3 of the property spec, is a property object is implemented by a supplementary class generated by the compiler. This means the compiler must create one class by property and so bloat the application with lot of stupid code.
Ok, let's try to do better. No class at compile time means reflection or runtime time generation. In fact, it's not a real choice because reflection since 1.4.1 generates byte-code when a method is often called.

Da Vinci VM

The Da Vinci VM is the prototype implementation of JSR 292, a modified hotspot VM patched with new entry points that help to implement dynamic languages on top of the Java platform.
The first feature available, anonymous class (VM anonymous class not compiler one) allows you to create classes with some interesting features:

1. The class is not referenced by a classloader, so is GCed when there is no more instances of that class
2. Class is like an inner-class of an host class and therefore can access to members of the host class.
3. In order to create the class you can patch an existing class.

How property object works

If you are not familiar with properties, you can read an old blog entry.
A property object stands for a property and permit access to the value of the property of any instance of a Class that declare that property.
By example, the following code:

public class Bean {
  private int x;
  public int getX() {
    return x;
  }
  public void setX(int x) {
    this.x = x;
  }
  
  private String text;
  public String getText() {
    return text;
  }
  public void setText(String text) {
    this.text = text;
  }
    
  public static void main(String[] args) {
      Bean bean = new Bean();
      Property propertyX =
          Property.create(Bean.class, "x");
      bean.setX(0xCAFEBABE);
      System.out.printf(propertyX + " %x\n", propertyX.getValue(bean));
      
      Property propertyText = Property.create(Bean.class, "text");
      propertyText.setValue(bean, "hello property");
      System.out.println(bean.getText());
  }
}

prints:

  int Bean.x cafebabe
  hello property

How to test it ?

If your OS is Windows, Solaris or MacOs you have to compile the VM by yourself, sorry. On linux, you can download the binary of the jdk7 b24 if you haven't already it, download the following zip davinci.zip and unzip it in your JAVA_HOME/jre/lib/i386/
It adds a new server JVM lib named 'davinci' that can be launched using

  java -davinci ...
 

Now download the property runtime support property-runtime.jar, because it adds new classes and overrides some existing classes to the JDK you must prepend the jar to the bootclasspath.
To launch the example above:

  java -davinci -Xbootclasspath/p:property.jar Bean

The source of the property runtime support are available on the kijaro web site here: https://kijaro.dev.java.net/source/browse/kijaro/branches/properties/.

What's the next step, finish to write a new version of the property spec and provide a modified compiler according to that spec. I think i've solved most on the corner cases, so it's just a matter of time.
To be continued...
Rémi

Sorry but i've toasted your pet

Introduction

The idea is simple. I want to create a calendar service, like the server part of Google Calendar. I want to create a server that is able to parse a specific protocol allowing to query calendars and send a response using by example the ical format.
This service can be queried by different languages so i've decided to use a text based protocol, like HTTP.
By example the request to get a calendar named mycalendar as user forax between November, the 6th and November, the 7th will be something like that.

GET mycalendar forax CAL/1.0
password: AjxHKRFkRwxx3j9lM2HMow==
from: Sun Nov  6 08:49:37 1994
to: Mon Nov  7 12:34:31 1994

More formally, requests can be described by the following grammar using the Tatoo EBNF form.
Tatoo EBNF form use sections:

• section tokens declares terminals and their regex.
• section blanks declares the regex whom the lexer will not send a terminal to the parser.
• section productions declares sequences of terminal and non terminal which will be reduced to a non terminal.

tokens:
  service='GET'
  uri= '([^ ])+'
  user= '([a-z][A-Z])+'
  protocol= 'CAL/1.0'
  colon= ':'
  header_key= '([^ :\r\n])+'
  header_value= '[^ \r\n]([^\r\n])+'
  eoln= '(\r)?\n'
 
blanks:
 space= "( |\t)+"

productions:
 start = request+
       ;
 request = firstline 'eoln' header* 'eoln'
         ;
 firstline = 'service' 'uri' 'user' 'protocol'
           ;
 header = 'header_key' 'colon' 'header_value' 'eoln'
        ;

Now Tatoo, a parser generator we have developed is able to create a non-blocking push lexer/parser for that grammar.
By non-blocking, i mean a parser that work on NIO buffers, and since christmas, Tatoo has a companion named banzai, a server using NIO that can embed a Tatoo parser generator.
Great, but how to specify the semantics, i.e how to specify that the password need to be checked, how the response is computed etc.

The semantics

The semantics is specified by creating a class that inherits from ProtocolHandler. A ProtocolHandler is an object called each time a terminal is recognized (shifted) or a production is reduced. Furthermore, it provides an object that own some methods like asyncWrite() to send back data to the client or endRequest() to end the request.

 public class CalendarProtocolHandler extends ProtocolHandler { 
  private String uri;
  private String username;
  private String header_key;
  private String header_value;
  private HashMap headerMap=...
  ...

  public void shift(RuleEnum rule,TerminalEnum terminal) {
    switch(terminal) {
      case uri:
        uri=decode();
        return;
      case user:
        user=decode();
        return;
      case header_key:
        header_key=decode();
        return;
      case header_value:
        header_value=decode();
        return;
    }
  }

  public void reduce(ProductionEnum production) {
    if (production==ProductionEnum.header) {
      headerMap.put(header_key,header_value);
      return;
    }
    if (production==ProductionEnum.request) {
      handle();
      return;
    }
  }
  
  private void handle() {
    try {
      String password=headerMap.get("password");
      boolean passwdOK=verifyPassword(username,password);
      if (!passwdOK) {
        putUnauthorizedResponse(outBuffer);
        analyzer.asyncWrite(outBuffer);
        return;
      }
      ...
      Calendar c=...
      putICalendar(outBuffer,c);
      analyzer.asyncWrite(outBuffer);
    
    } catch(IOException e) {
      analyzer.endRequest(e);
    }
  }
}

Short explanation of the code above: shift(), if a specific terminal is found, decode the buffer to find its value. reduce() if we reduce a header key/value pair, store it into a map, if we readuce a request, call handle(). handle() verify the password and write a response

Banzaï !

I know what you are thinking:
"Ok, it's interresting. But embeding a parser in a webserver will damage performance".
No, Tatoo parser is carefully designed to not hurt performance.
I know that you don't believe me. So let me try to convince you with a stupid benchmark(tm).
I have written a subset of the HTTP/1.1 grammar (a subset because the wole HTTP/1.1 grammar is huge and i'm lenient), generated the corresponding parser with Tatoo, written a ProtocolHandler corresponding to HTTP and beanchmark banzai (the server) with that protocol handler and comparing it with Grizzly and Jetty.

Benchmarks

So i've borrowed two DELLs (config) with ethernet Gigabit cards and a gigabit switch in my labs, plug them and play.
Servers are set up with a Gentoo (2.6.19-gentoo-r5) without major modifications. I have just raised the number of descriptors to 65535.
Because i want to do a stress test, i've used apache bench (ab) as client so this is not a real scenario, just a stupid benchmark(tm).
If you want to reproduce the test, checkout the code here svn checkout https://svnigm.univ-mlv.fr/svn/tatoo/trunk, compile Tatoo using ant.
Banzai is located in a sample directory named httpserver. use ant all compile to compile banzai, and to launch it using

 java -server -cp classes:../../lib/tatoo-runtime.jar fr.umlv.tatoo.samples.httpserver.banzai.Main

First test: how many requests can be handled by banzai

Serving files of 4k, 8k, 16k and 32k with different numbers of concurrent connections (8, 16, 32 etc.). The value is a mean over 25 runs of 50 000 requests.
First, when i have seen the result of the benchmark I was astonished. Wow, linux 2.6 and a Core Duo offers great perf, more than 25 000 requests by second for a 4k file.
Furthermore, it seems that banzai have a problem if there are lot of concurrent connections. Perhaps because banzai doesn't have a strategy like closing keepalive connections if there are lot of connections.
Now, a more interesting graphs.

Second test: comparing with the others

Serving a 4k with different concurrent connections
Yes, banzai performs slightly better than grizzly ...
at least if there are not lot of concurrent connections.
About Jetty, I think I screw up the conf because the slope of the curve is weird and I am not able to explain why.

What's next

I think it's possible to integrate non blocking parsers technology directly into grizzly.
Another idea is to wrap banzai to use RESTlet API.
To end, i have found small improvements to the OpenJDK and i will send patches that will benefit to anyone.

Cheers

CICE prototype available and FOSDEM

It's an old news but i've just discovered that Mark Mahieu provide an implementation of CICE closure proposal which is an aternative to BGGA prototype.

By the way, i will be at FOSDEM'08, if you want to meet me, i will try to attend to all Free Java Meetings.

I have decided to finish this entry a la Chris Campbell.
In my ears: Morcheeba, "Dive Deep"
In my eyes: Some cryptic codes like always.

Yet another closure proposal

Everybody comes with its own closure proposal, why not me :)

Unlike BGGA, CICE, or FCM (not entirely sure about FCM), i don't like the fact a closure is an instance of a class that implement a method. I prefer the John Rose's vision, closure are at runtime method handler on autonomous block of code.

There is another thing I don't like in the current BGGA proposal: its syntax of function type, {int,int=>int} is not enough Javaish for me. My proposal comes with a new way to declare a function type

I've separated my proposal in three parts:

1. function type declaration
2. how to use function type
3. closure syntax

Function types declaration

A function type is declared using the 'almost' keyword function.

// function types declaration
class Ops {

  // function are basically like typedef
  // top-level functions are illegal
  function <T> int comparator(T value1,T value2);

  // here function is not a real keyword, the compiler
  // recognize it as a keyword only in this construction
  function <E> void block(E element);

  // in the function below, E can be instantiated to any
  // object types or any primitive types,
  // it works because function are only declaration with no code. 

  // another function
  function <R> R task();
}

How to use function type

Function types are types so they can be used anywhere a type can be used except as bound of a type variable or type argument of a type variable.

// usage
import static Ops.*;

class Utils {
  // note: comparator<?> is illegal
  // (wildcard are prohibited for generics functions because
  // they are not needed).
  static <T> void sort(List<T> list,comparator<T> comparator) {
    ...
  }

  static <T> void forEach(Collection<T> c,block<T> block) {
    for(T t:c) {
      // invoke is a magic method that execute the block of code,
      // a special rule in the compiler is able to infer its
      // parameter type.
      block.invoke(t);
    }
  }
}

Closure syntax

Unlike function type syntax of BGGA, i like its closure syntax so basically my proposed closure syntax is identical. I have borrow method references from FCM even if we need to add a special lookup rule (because methods and fields are not in the same lookup space) for that in the compiler.
Like BGGA, my proposal include non local transfer and access to local variable (read and write) but only for closures and not for any anonymous classes, a closure is not an anonymous class. Like any proposal, my proposal allow interface conversion of function to interropt with legacy codes but only if closure use final local variable.
At last, return is not allowed in a closure to avoid stupid puzzlers but a replacement syntax exists.

// at calling site
class Main {
  static int f(Object o1,Object o2) {
    ...
  }

  void g(int value) {
    ...
  }

  public static void main(String[] args) {
    Ops.comparator<Object> c=Main.f;

    // using a primitive type here is legal,
    // it creates a block of code that call g()
    Ops.block<int> b=new Main().g; 

    List<String> l=Arrays.asList(args);
    Utils.sort(l,c); // ok contravariance

    // closure syntax, this syntax allow to specify the block of code
    Utils.sort(l,{Charsequence c1, CharSequence c2 =>
      c1.toString().compareTo(c1.toString());
    });

    // function to interface conversion
    java.util.Comparator<Object> javaUtilComparator=c;

    // it's roughly equivalent to
    final Ops.comparator<Object> tempc=c;
    java.util.Comparator<Object> javaUtilComparator=new java.util.Comparator<Object>() {
       public int compare(Object o1,Object o2) {
         return tempc.invoke(o1,o2);
       }
    };
    
    // closure to interface conversion
    java.util.Comparator<String> javaUtilComparator2={String s1, String s2 =>
      -s1.compareTo(s2);
    }); 
  }

Examples of non-local transfer and local variable mutation.

  // this is not allowed in contrast to BGGA.
  class NotAllowed implements Ops.block {
    ...
  }

  static int counter(Collection<Integer> c) {
    int count=0;
    // bloc of code (closure) can mutate local variable,
    // a function that returns an int is a subtype of a
    // function that returns void.
    Utils.forEach(c, {Integer i=>
      count+=i;
    });
    return count;
  }

  static boolean startsWith(Collection<String> c, String prefix) {
    // closure with non local tranfer,
    // moreover prefix doesn't need to be final
    // because this is not an interface conversion
    Utils.forEach(c, {String s=>
      if (s.startsWith(prefix) {
        return true; // doesn't compile ambiguous
        contains.return true; // ok
      } 
    });
  }
}

I wait your comments,
Rémi

Java 7 - Extension methods

Recently, Neals, Peter and Stephen blog about extension methods.

Why i hate (yes hate) use-site extension ?

What i love (yes love) with Java is the fact that i can take a look to a screen over one of my student shoulder and be able to say is the snippet i see is correct or not.
With the use-site extension proposed by Neal, the behavior of a code depends on some static imports that are located in the beginning of the file. So you can't understand a code without browsing the entire file.

Demonstration. Do you understand this code ?

 ...
 String text=...
 for(int i=0;i<text.size();i++) {
   System.out.println(text.charAt(i));
 }

Hum, i have forget to say that the begining of the file contains that declaration: import static Strings. and that the class Strings is defined like that:

 public class Strings {
   public static int size(String s) {
     return s.trim().length();
   }
 }

I'am sure i will hate that.
It will remember me the the mess of C++ member functions: "where is the f***ing def of this method".

Furthermore, import static has already some limitations, the code below doesn't compile. This case is currently rare but with extension methods it will be more frequent.

  import static java.util.Arrays.*;

  public class StaticImport {
    public static void main(String[] args) {
      toString(args);
    }
  }

Why i prefer declaration-site extension methods ?

It's definitely a good property of a language to be able to understand a code that deals with some java.util.Lists, just by knowing the declaration of this type. If i don't know how to use it, i can just to browse its code. After that, because i have a good brain storage device, i will be able to understand ALL codes that deals with java.util.List.

Another proposed syntax

I think there is some problems with the syntax proposed by Peter.

  package java.util;
  public interface List<E> … {
  …
    void sort() import static java.util.Collections.sort;
  …
  }
 

1. A new syntax, again !
2. The syntax let a developer think that this method can be overriden but without a modified VM it's not possible.
3. The constraint on the elements of the list (must be comparable) are not visible.

Here is my proposed syntax, allow to declare static method in interfaces (in the same time, we close one the top 25 RFE Bug 4093687) and add some sugar in the compiler.

  package java.util;
  public interface List<E> … {
  …
    @ExtensionMethod
    static <T extends Comparable<? super T>> void sort(List<T> list) {
      java.util.Collections.sort(list);
    }
  …
  }

Like other proposed syntax, the compiler is modified to transform an instance call to a static call. The following code

  List<String> list=...
  list.sort();
 

  List<String> list=...
  List.sort(list);
 

@ExtensionMethod (borrowed from Stephen blog) like @Override is not mandatory and tells the compiler to verify that the first parameter of the static method is the same type that the declaring interface.

I wait your comments,
Rémi

Resurrect dolphin

This morning is was in my shower, thinking about life, sea etc, when the word 'dolphin' pop in my mind.
It instantly remember that day, when SUN management decide to stop using codenames.

I am certainly a geek but i was shocked and, for me, this day will always be a day in gray. Afterall, SUN decide to kill dolphin.
I always think it's a little bit easier to try to explain to anyone that you have spent a whole night on ladybird than on jdk7 :)

I think it's time to resurrect the dolphin. Now, the openjdk is an opensource project, lead by a community not by some managers and lot of opensource softwares have a codename like Gran Paradiso, Etch, Moonshine, etc.

So i officialy request to be able to use dolphin as codename for openjdk7.

I wait your comment and hope your support.

Rémi

Mixing property language support and bean bindings

Recently, Shannon posts version 1.0 of beanbindings, even if this version is not ready for production use, it is stable enough to create a small demo mixing property language support and beanbindings.

The idea is to create a bean (here MyBean) with a bound property (label) and to bind this property to two different textfields.

Defining a bean

Because i use the property support directly integrated in the language, defining such bean is easy. AbstractBean provides the basic support for managing property change listeners.

 public static class MyBean extends AbstractBean {
    property String label bound;
    
    private <B,T> void propertyChanged(java.lang.Property<B,T> property,Object oldValue,Object newValue) {
      firePropertyChange(property,oldValue,newValue);
    }
    
    @Override
    public String toString() {
      return "label "+label;
    }
  }

Wrap the property to someting understandable

By default bean bindings only supports two kinds of property bean property that uses bean introspector and EL property that uses EL script but the framework is easily extensible and let you create your own property implementation.

So i've created a new kind of property (here LangProperty) that exposes a java.lang.Property as a property understandable by the bindings framework.

And mix

And now mixing the language support and the API together is as simple as that:

    LangProperty<MyBean,String> labelProperty =
        LangProperty.create(MyBean#label);
    
    JLabel label = new JLabel("Label:");
    JTextField field = new JTextField(20);
    Bindings.createAutoBinding(READ_WRITE, 
        bean, labelProperty,
        field, BeanProperty.create("text")).bind();
    
    JLabel anotherLabel = new JLabel("Another label:");
    JTextField anotherField = new JTextField(20);
    Bindings.createAutoBinding(READ_WRITE, 
        bean, labelProperty,
        anotherField, BeanProperty.create("text")).bind();

The notation sharp ('#') is used to get an objet typed java.lang.Property that allow to get and set the value of the property.

Want to test it

You want to test it by yourself, ok, here a zip containing the code, the beanbindings jar and the javac.jar patched to understand properties.
Download properties-meets-bindings.zip

Use this line to compile

 java -Xbootclasspath/p:javac.jar com.sun.tools.javac.Main -XDallowProperty -cp beansbinding-1.0.jar *.java

And this one to execute

 java -Xbootclasspath/p:javac.jar -cp .:beansbinding-1.0.jar Sample

You must include javac.jar at runtime only because it contains the class java.lang.Property and i am too lazy to create two jars.

WARNING, i have re-written the bound property support in the patched compiler last night so i think it will not work on another example :)

Cheers,
Rémi

Java kernel in jdk6 update 4

From bug report Bug 6585322 it seems that Java Browser Edition, Java Kernel will be available in jdk6 update 4.

Cheers,
Rémi

Beansbinding goes to the wrong direction

Shannon Hickey recently post a blog entry about a new release of Beans Binding project at http://beansbinding.dev.java.net
I have take a look to the documentation and my first belief is that the API doesn't guide the user enough.
But let me first introduce the concept of bindings.

What is a binding ?

A binding is a two-way connection between two properties (javabean property). When you change the value of one, automatically the value of the other one is changed.
The concept is not new, i have used JGoodies binding two years ago and the framework was mature at that time. And this concept is heavily used in web tier like Tapestry, JSF etc.

Why do we need bindings ?

Bindings are very elegant solution when you want to cleanly separate the presentation logic to the presentation view of a form.
The presentation view is a panel or a dialog with its components and the presentation logic is a bean. A binding allows you to connect a component (textfield, combo, etc) property to its corresponding bean's property.
So by example if you decide to change a textfield by a combo, you only update the presentation view, and doesn't change the presentation logic.

Furthermore bindings are great because:

1. it provides a unified concept that hide the complexity of dealing with different swing listeners, by example, the selected value of a list is not a bound property, a textfield can't be changed when it fire a modification event, etc.
2. if you set a validator to a binding, you can do validation (on the fly), and provide a feedback to the user by changing the component visual aspect. Futhermore if you have a way to group bindings you can easily find the list of all unvalidated bindings.
3. they can manage conversions (by example between text and primitive type, date etc) because a bean property exports its type.
4. dealing with swing listeners often leads to memory leaks, with a binding you can easily remove listeners by unbinding it or better by unbinding a group of bindings.

If you want more info, take a look to an introduction to jgoodies binding by R.J. Lorimer.

Simplifying the MVC

beansbinding spec/implementation allow not only to define bindings between simple components but to define bindings on MVC components like JList, JTable, JTree, i.e on components that display multiple objects. This provide a more simpler way to define component's model.

Let me take an example, i have a list of employees and i want to display their names in a JList. Moreover, if i modify the employee list or if i change the name of one employee, i want the JList to reflect that change.
A JList (like JTree or JTable) has 3 kinds of events, added, removed or changed. So my list need to be able to signal addition or removal and employee objet need to signal when its name changed.
Employee name modification is signaled using a classical binding

  List list=...
  new Binding(list, null, "${name}");
 

Why do we need the Expression Language ?

beansbinding let you specify the bound property using a enhanced version of the Expression Language (EL) script already use in JSP or JSF.
Suppose that now i want to display the list of street name of my employee. I can create this kind of binding

  List list=...
  new Binding(list, null, "${address.streetname}");
 

So if binding are is a great concept why don't you like beansbinding ?

I just don't like some parts of the current implementation of bean bindings:

1. javax.beans.binding.Binding is not an abstract class, so it's not a concept. I can't create a binding on a user defined component that have a specific listener.
2. beansbinding allows to define a tree of binding but i think it's not necessary, a path of bindings (or a collection/map of binding path) is sufficient.
3. you can define a binding and forget it, this will lead to memory leaks, All bindings need to appear in a group of bindings to be unbound easily.
4. name a binding (new in 0.6) is not a good solution(tm), an object is always more powerfull than a string, if you want a reference to a binding, store it in a field.
5. there is no simple way to get the list of bindings that have validation errors. Furthermore, pratically convertions and validations are often the same operation in Java, by example there is no method Integer.isAnInteger(), Integer.parseInt() do the convertion or throw an exception.
6. there is no way to set a collection of possible conversions for a group of bindings.
7. beanbindings use EL script so goodbye type safety. There is a real risk of creating a desktop applications that will work or not depending on inputs (by example, of data of a database) Java really needs something like C# expression trees, a part of (D)LINQ.
8. the way to define bindings for MVC component is clumsy, by example you can set the parameter of a tree to a binding that bind a table.
9. there is no support for dynamic bean. Roughtly, a dynamic bean is a hashmap of String/value with a property change listener mecanism. Swing's AbstractAction is that kind of bean, so there is currently no way to bind the name of an action to a text of a textfield.

Proposed new Design

Ok, i ve tried to summarize all my remarks in a new design:

  public abstract class Binding {
    Object getSource();
    Object getTarget();
    Binding setValidator(Validator validator);
    void bind();
    void unbind();
    //etc...
  }

  //BindingContext renamed
  public class BindingGroup {
    List bindingList() {...}
    void bindAll() {...}
    void unbindAll() {...}
    // list of all bindings that are not validated
    List unvalidatedBindingList() {..}
    void addValidationListener(ValidationListener l) {...}
    void removeValidationListener(ValidationListener l) {...}
    // the mapper find a converter by its source and target types
    void setConverterMapper(ConverterMapper mapper)
  }

  public interface ValidationListener extends java.util.EventListener {
     void validationFailed(Binding binding, ValidationResult result);
     // allows to remove visual effect on component
     void validationOk(Binding binding);
  }

  // i have tried to separate bindingroup concept from 
  // the binding factory, not sure that is a good idea
  public class BeanBinder {
    public BeanBinder(BindingGroup group) {...}
    public BeanBinder() { this(new BindingGroup()); }
    public BindingGroup getBindingGroup() {...}
    public Binding addBinding(Object source,String sourceExpr,Object target,Object targetPath) {...}
  }
  
  // add swing bindings for some specific components
  public class SwingBinder extends BeanBinder {
    public Binding addSelectedBinding(JTextCompmonent text, Object target,Object targetPath) {...}
    public Binding addSelectedBinding(AbstractButton button, Object target,Object targetPath) {...}
    public Binding addBinding(AbstractAction action String sourceExpr, Object target,Object targetPath) {...}
    public Binding addSelectedBinding(JList list, Object target,Object targetPath) {...}
    //etc.

    ListBinding addListBinding(JList list, ObservableList list) {...}
    ListBinding addListBinding(JList list, Object source, String sourceExpr) {...}

    TableBinding addTableBinding(JTable table, ObservableList list) {...}
    TableBinding addTableBinding(JTable table, Object source, String sourceExpr)
{...}    
    TreeBinding addTreeBinding(JTree tree, Object root) {...}
    //etc.  
  }
  
  // list specific binding
  abstract class ListBinding extends Binding {
    JList getSource() {...}
    ListModel getModel() {...}
    Binding addElementBinding(String elementPath) {...}
  }

  abstract class TableBinding extends Binding {
    JTable getSource() {...}
    TableModel getModel() {...}
    Binding addColumnBinding(int index,String elementPath) {...}
  }

  abstract class TreeBinding extends Binding {
    JTree getSource() {...}
    TreeModel getModel() {...}
    Binding addNodeBinding(Class nodeClass,String elementPath) {...}
  }
 

What do you think about that ? i wait your answers.

Cheers,
Rémi

Java property (draft 3)

This is a new step in my quest (or curse) to provide properties to Java.
I've written a new version of the proposal (the third draft) of the property spec, available as a google doc
property draft (v3)

During the last month, i've written a new prototype from scratch that follows the specification. The prototype is not available yet, mostly because i change the spec too frequently, but i hope to post a blog about its availability in one or two weeks.

If you are concerned by the implementation details of the prototype, i will soon post messages that describe its implementation to Kitchen Sink Language mailing list.

I wait your comment about the spec.

Cheers,
Rémi

@nnotation type or type @nnotation

The purpose of JSR 308 is to allow to define annotation on types.
Currently, the JLS 3 only allows to annotate language elements than accept modifiers so it's not possible to annotate types.

Why allowing this is a good idea ?

There is real interest to allow annotation on Java types, it enables to write safer code by performing static source code analysis. Imagine annotations like:

1. JetBrain's IDEA @NonNull, @Nullable to avoid NullPointerException and helps database mapper
2. @Scoped to denote objects allocated in scoped memory of RTSJ
3. Java Concurrent In Practice @Immutable or @ThreadSafe that allows to write safer code in a multi-threaded environment.

There are two possible syntax, the first one is to allow annotation of type at the left of the type like any other Java annotations. But it introduce an ambiguity because a reader doesn't see clearly if an annotation belong to a variable, a method etc. or its type. In the following sample, @NonNull refers to String and @Column refers to name.

  @Local
  class MyBean {
      @NonNull @Column(name="_NAME") String name;
      @Deprecated @NonNull Dimension getSize() { ... }
  }

The second one is to allow annotation on type at the right of a type, in that case there is no ambiguity.

  @Local
  class MyBean {
      @Column(name="_NAME") String @NonNull name;
      @Deprecated Dimension @NonNull getSize() { ... }
  }

The JSR308 expert group seems to think that the first syntax is better, more Java-like even if it introduce an ambiguity.
I am not totally convinced.

What do you think ?

Cheers, Rémi

Closure Litteral and Method Reference

Recently, Stephen Colebourne and Stefan Schulz post another closure like proposal, yes, yet another one.

They propose another syntax for describing a closure which, in my opinion, is more a simpler way to declare an inner-class.
I am not a big fan of this proposal, i definitively prefer the closure syntax decribed by Neal Gafter.
But there is something that i like in their proposal, the fact that you can create invocable method references.

Invocable Method Reference

We need a way to easily create a reference to a method, an object that allow to call a method. Currently a method is not an Object and if you want something like that you have to use reflection (java.lang.reflect.Method) which is not type-safe, uses lot of checked exceptions and performs primitive boxing and array boxing.

So having a way to create a method reference is a good think. But unlike Stephen and Stefan, i don't think that a method reference is a java.lang.reflect.Method correctly typed by the compiler but instead a java.function object (a closure) of Neal.
In my opinion method reference is a kind of closure litteral, so this code snippet:

  public void init() {
    JButton button = ...;
    button.addActionListener(this#handleAction(ActionEvent));
  }
  public void handleAction(ActionEvent ev) {
    // handle event
  }

is a short syntax for:

  public void init() {
    JButton button = ...;
    button.addActionListener({ActionEvent e=>
      handleAction(e);
    });
  }
  public void handleAction(ActionEvent ev) {
    // handle event
  }

Because a method reference is a closure litteral, the type of a method reference is a function type.

 {ActionEvent=>void} handle = this#handleAction(ActionEvent);

I think that using # is a good idea. Perhaps because it's the syntax i choose to use to create property litteral (property reference) in my next property proposal that i will post in few days.

Cheers, Rémi

Meet me at FOSDEM

If you want to talk about:

1. Why the JLS doesn't allow array of parametrized type.
2. Kitchen Sink Language.
3. Property syntax.
4. Why beers are so great in Belgium ?

I will be saturday and sunday at FOSDEM.

Rémi

Property Reloaded

This entry is the second draft of my property proposal, i have tried to gather all the ideas proposed since my first post about properties.

Why do we need a property syntax in Java ?

If you have not followed the different blogs, i recommend you to first read properties in Java by Richard Blair and Properties are design features by Cay Horstmann.

Declare a property

There is three kind of property, simple property that are translated to a field plus a getter and a setter automatically generated by the compiler, user-defined property, a property for which getter and setter are defined by the developer and abstract property, a property for which getter and setter are abstract. All kind of properties can be read-only, write-only or read-write.

  class MyBean {
    public property String name1; // read-write
    public property String name2 get; // read-only
    public property String name3 set; // write-only
    public property String name4 get set; // illegal

    public property String name5 get { return "hello"; } // read-only
    public property String name6 set(String name) { } // write-only
    private String _name7;
    public property String name7
      get { return _name7; }
      set(String name) { _name7 = name; firePropertyChange(name7); } // read-write
  }

As i have already said, property, get and set are not real keywords but only local keywords. A property is implicitly read-write so the syntax for name4 is illegal. All other combinations that the ones above are invalid. You can also notice that if the property is not abstract and ends with a semi-colon, a field is created.

Access to a property

Unlike my first proposal, i think now that we must not have two ways to access to a property. So 'dot' can't be used to access/change the value of a property (It has another meaning, see further). In order to permit retrofitting of already existing code, properties are accessed using getXXX/setXXX. It's a little awkward especially for beginners but i really think that is the best solution.

 MyBean bean = new MyBean();
 bean.setName1(bean.getName2());

Property Litteral

Because i'm lazy, instead of defending myself property literal, i prefer to redirect you to Evan Summers's property reference or Stephen Colebourne's property objects
Matthias Ernst proposed to create a property object by property and by object. I think it is not very feasible due to the memory comsumption. I think a system ala java.lang.reflection, i.e. one property object by property and not related to an instance is more realisic.

I propose to use dot on the class to access to such object.

  Bean bean = new Bean("joe", "forax"); 
  Property<Bean, String> name = Bean.name;
  name.set(bean, "rémi");
  Property<Bean, String> lastname = Bean.lastname;
  String myname= lastname.get(bean);

A property literal is like .class, it acts like a static final field. The compiler translates ClassName.propertyName to ClassName.class.getProperty("propertyName").
Perhaps the property object can be cached to avoid one lookup by call like .class in transalted by the compiler in pre-tiger release.

Property Litteral and generics

The idea is to make Property safe, so a property is typed using generics. I propose to parametrize a property objet Property by its bean type and the type of its value.

 Property<Bean, String> prop = Bean.name;

Bound Property

A simple property can be bound to make swing hackers happy. If such property is defined in the class, a method propertyChange() is required in the class or in on of its supertypes. A bound property is always read-write.

 class Point {
    public property int x bound;
    public property int y bound;

    private <T> void propertyChange(Property<Point, T> prop, T oldValue, T newValue) {
      // do what you want here
    }
 }

The compiler translates the bound property x into :

 private int x;
 public int getX() {
   return x;
 }
 public void setX(int x) {
   int oldX=this.x;
   if (x!=oldX) {
     this.x=x;
     propertyChange(Point.x, oldX, x); 
   }
 }

Property Litteral and switch

Property litteral like enums can be used in a switch. This features is easy to implement if bug 5029289 (switch on strings) is implemented.

 Bean bean=new Bean();
 bean.addPropertyListener(new PropertyListener<Bean>() {
   public <T> propertyChanged(Property<Bean, T> property, T oldValue, T newValue) {
     switch(property) {
       case name:
         frame.setTitle((String)newValue);
         break;
       case value:
         slider.setValue((Integer)newValue);
         break;
     }
   }
 }

Like enums, case items are not qualified. The compiler verifies that as the variable property is typed Property<? extends Bean, T> the properties Bean.name and Bean.value exist.

The class Property

Just a preview of the methods of the class Property.

  public class Property<B, T> {
    public Class<B> getDeclaringClass() {}
    public Class<T> getType() {}
    public String getName() {}
    public boolean isReadable() {}
    public boolean isWritable() {}
    public T get(Object bean) {}
    public void set(Object bean, Object value) {}
  }

  public class Class<T> {
    public Property<T, ?>[] getProperties() {}
    public Property<T, ?> getProperty(String name) {}
  }

The signature of getProperties() is illegal with the current JLS but i hope it will be legal with the next JLS.

I will implement this spec functions of your comment, cheers
Rémi

Kitchen Sink Language and local keywords

The Kitchen Sink Language (or ksl) is open here : ksl.dev.java.net, so i hope that the next version of my prototype will be the ksl trunk.

Before trying to commit compiler patches that introduce local variable type inference or properties to the ksl trunk, i think i will write a small patch that enable the use of local keywords.

Local keywords

What is a local keyword ? A local keyword is a keyword that is only enabled at some locations in the grammar and that is considered as an identifier elsewhere.
It will ease anyone to introduce new keywords like property, indexer, dynamic or <insert your favorite keyword here>.

Just a word about properties, i have read lot of blogs about that, i have even dreamed about that this week end (a kind a nightmare in wich a property told me that she needs to be bound). So i have decided to take some rest.

Rémi

Property and interceptors

Since my last post, Cay Horstmann has recalled that properties are intended for tools and Hans Muller bid up by saying that property syntax is useless without a way to define bound properties.

Interceptors and bound properties

What Hans want is some kind of interceptor. An interceptor is an object or a method that can trap access to a property. In Java, these objects are implemented either using java.lang.reflect.Proxy or by bytecode enhancement using a VM agent or a special classloader.
It is interresting to know that bean interceptors are already defined in EJB 3 spec (JSR-220).

So for me, instead of trying to re-invent the wheel, we could perhaps transpose the concept of server side interceptors in Swing world. We 'just' need an equivalent to an EJB container for swing application. I think the concept of transaction can be borrow in the same time, it could liberate us from invokeLater and SwingWorker.

So in my opinion, adding bound properties to the language is a bad idea.
I prefer to let that job to Beans Binding experts group (JSR 295) and its leader Scott Violet.

So is property syntax usefull ?

I think that even if bound properties are not managed by the compiler, we need a property syntax at least to insert a special attribute in the bytecode that will be recognize by the reflection runtime to provide property objects at runtime.

Property interceptor by the compiler

But if you think that this kind of AOP must be done by the compiler, i propose to allow to declare two a special methods named set* and get* (the star is not a typo) that can be used instead of a particular getter or setter.

By example, a bean that declares two bound properties background and foreground in that way could be written like that :

  public class MyBean {
    public property Color background;
    public property Color foreground;

    public void set*(Property property,Object value) {
      Object oldValue=property.get(this);
      super();
      SwingPropertySupport.firePropertyChange(property,
        oldValue,property.get(this));
    }
  }

The compiler will generate foreground or background setters by duplicate the set* code for each setter.
java.lang.reflect.Property is the reflection counterpart of a property at runtime (I've just rename the Stephen Colebourne's property object to a more Java-like name), super() in that context means set the property value (pretty ugly isn't it) and SwingPropertySupport is a new class that allow to add/remove PropertyChangeListener for any property of any bean.

To Link a property to a listener, we can use

  MyBean bean=...
  SwingPropertySupport.addPropertyChangeListener(
    bean,"background",new PropertyChangeListener() {
      ...
    });

Cheers,
Rémi

P.S : to the little Peter, you can't return a gift :)

Property and bean spec

Among the questions about my property proposal and its implementation. One can be answered easily.

Why don't provide property change support ?

Because there are several ways to implement a property change support depending on what you want.

I've try to categorize them, conbinations are possible:

• Listener and event can be at least the following couple: PropertyChangeListener/PropertyChangeEvent, VetoableChangeListener/PropertyChangeEvent, an PropertyChangeListener,/IndexedPropertyChangeEvent, or the swing lightweight ChangeListener/ChangeEvent.
• Register/unregister listeners Listeners are stored in a collection, but which one ? A list or a set. Is the collection threadsafe, using synchronized blocks, locks or other concurrent mecanisms ? The traversal is in ascending, descending or any order.
• Listener and thread The listener method is called by the current thread, another thread (using by example an executor) or a special thread (like the EventDispatchThread).

I hope you are convinced.

All your property are belong to us

Happy new year everyone.
Since my last post, i've done some homeworks :),
and i'm please to present you a new version of the prototype java compiler which includes property support.

First, why properties ?, we have already fields and any IDE have a menu item that can generate getters and setters, so why do, we need properties ?

Why do we write getter and setter ? The reason, is that when you use a property, you don't want to know if its implementation use a field or something else. By example, you can first choose to implement it as a field and later when a new feature is require change your code to use a method. To be binary backward compatible, a carefull developer will always define a getter and a setter, even if the property is a field package visible, but its a boiler plate code and i don't see a reason why we should do that if the compiler ensure that a change in the way property is written will always be backward compatible.

The compiler must handle properties for you and garantee that implementation change must be binary compatible without writing boilerplate codes.

Defining a simple property

  public class PropertyPoint {
    property int x;
    property int y;
  }
  ...
  public static void main(String[] args) {
    PropertyPoint p = new PropertyPoint(1, 2);
    System.out.println(p.x + " " + p.y);  // replaced by p.getX(), p.getY()
  }

A word about the keyword 'property', 'property' is not a real keyword like 'public' or 'enum', so you can have a variable named property in another place in your code, i will still compile, even if you create a class property, it will still compile. Basically, if the compiler found the keyword 'property' in another place, it consider it as an identifier or a type depending on its location. This example, written by peter ahé, compiles :

  public class property { // see the name of the class ??
    public property int value; // a property
  
    // these stuffs are not properties
    property property;    // a field
    property() {          // a constructor
    }
    property property() { // a method
      return null;
    }
    property property(property property) { // another method
      return null;
    }
}

The compiler generates automatically a getter and a setter and doesn't allow developers to define their own. A property is accessed as a field (with dot) in the source code and by a getter/setter pair in the generated bytecode.

Abstract property

An abstract property is a property that is not a field thus can't be initialized by an expression. It requires the developer to write a getter and a setter. By example, i can tranform the class PropertyPoint to use polar coordinates instead of rectangular ones.

  import static java.lang.Math.*;
  public class PropertyPoint {
    abstract property double x;
    abstract property double y;
    
    property double rho;
    property double theta;
    
    public double getX() {
      return rho*cos(theta);
    }
    public void setX(double x) {
      double rho = hypot(x, y);
      theta = atan2(y, x);
      this.rho = rho;
    }
    
    public double getY() {
      return rho*sin(theta);
    }
    public void setY(double y) {
      double rho = hypot(x, y);
      theta = atan2(y, x);
      this.rho = rho;
    }
    
    public PropertyPoint(double x, double y) {
      this.x = x;
      this.y = y;
    }
}
...
    public static void main(String[] args) {
      PropertyPoint p = new PropertyPoint(1, 2);
      System.out.println(p.x + " " + p.y);  
    }

You can notice that the source of the method main doesn't change et even better, the generated bytecode doesn't change too. Another example, that mix abstract property and field :

  class Button {
    private String label;
    public abstract property String text;
    public String getText() {
      return label;
    }
    public void setText(String label {
      this.label = label;
      repaint();
    }
  }
  ...
  public static void main(String[] args) {
    ...
    Button button = new Button();
    button.text = "toto"; // replaced by button.setText("toto");
  }

Property and type

It is possible to define a property in a class or an abstract class, an interface (automatically public abstract) or an enum, but not in an annotation (currently not enforced by the prototype).
Allowed keywords for property are :
private protected public final static volatile transient.
Allowed keywords for abstract property are :
abstract private protected public synchronized final static

The current prototype has the following limitations:

1. properties are not recognized in an already compiled code.
2. final property support is untested.
3. abstract property doesn't check if a getter/setter exist in the hierarchy but only in the current class.
4. property doesn't support ':=' initialization

1. using a property in the constructor leads to an unsafe publication of this.
2. rules about overriding property are not defined nor the reflection API

The patch againt Open JDK compiler 1.7b5:
patch-1.7b05.txt
The prototype compiler:
prototype-1.7-b05.jar

How to use the prototype ?

To enable the property syntax, call javac in this way:
java -jar prototype-1.7-b05.jar -XDallowProperty Test.java

cheers, Rémi

Continue Reading...

Call me Santa

Since my last post, i've played with javac enought to be able to provide a patch that enables to let the compiler infers the type of local variables.

Life is a matter of choices

If you have already read the last Peter Ahé's blog entry you know that, at least, two proposed syntaxes compete.
The first one suggested by James Gosling and named 'Algol' in the prototype use ':=' instead of '=' to declare local variables. I've tweak the example widely used to explain generics and autoboxing without declaring the type of the local variables.

  // print a frequency table of words
  public static void main(String[] args) {
    map:=new HashMap<String,Integer>();
    for(word:args) {
      freq:=map.get(word);
      map.put(word,(freq==null)?1:freq+1);
    }
    System.out.println(map);
  }

Note that the compiler infers local variable in foreach loop too.

The second syntax supported by Peter Ahé and Christian Plesner Hansen use the keyword final to acheive the same goal. My running example with the 'final' syntax:

  public static void main(String[] args) {
    final map=new HashMap<String,Integer>();
    for(final word:args) {
      final freq=map.get(word);
      map.put(word,(freq==null)?1:freq+1);
    }
    System.out.println(map);
  }

How to use the prototype ?

To enable the Algol syntax, call javac in this way:
java -jar prototype-1.7-b04.jar -XDinferLocally=Algol TestAlgol.java

or if you prefer the final syntax:
java -jar prototype-1.7-b04.jar -XDinferLocally=final TestFinal.java

What you can do ?

You can test the prototype and let us know what you thinking.
Do you prefer the Algol syntax, the final syntax or none of both (perhaps because you hate the idea behind) ?

Download the prototype : prototype-1.7-b04.jar
Download the prototype patch against the Open JDK Compiler (1.7b04): patch-1.7b04

Cheers, Rémi

Type inference of local variables

For an average duke, the big difference between a language like PHP, Python and Java is that you have to declare the type of the variables. In general, it's not a big beal for a statement like this one

  foo=new Foo();
 

  Foo foo=new Foo();
 

But with the introduction of generics, a creation of collection like this one

  HashMap<String, List<Account>> accounts=
    new HashMap<String, List<Account>>();

leads to a stupid duplication of lot of characters and obfuscate the code.

I'm not the first to write about that, this was discussed when generics was introduced (see by example an artima forum thread about generics and the DRY principle)
Today we have at least two concurrent proposals to avoid to write a type twice: shorter instance creation or type inference of local variable.

Shorter instance creation

The first one come from the last blog entry of Danny Coward about jdk 1.7, in slide 38 under the title "shorter instance creation", you can find this example:

 LinkedList<String> list=new();

The type used by new is the type of the declared variable. I'm not a big fan of this proposal because it is focus on object creation and not on variable declaration. This proposal doesn't simplify a JEE sample like this one:

  List<Employee> employeeList=
    entityManager.findAll(Employee.class);

Type inference of local variables

The other proposal is to add a way to declare a local variable without explicitely mention its type and let the compiler infer it using the right hand side of the assignation.
This second poposal is not new, i've found an old RFE in the database that describes it : bug 4459053. This RFE was closed but i was before tiger so before the JLS3 describes how a type can be infered from an expression.
The RFE proposes to add a keyword 'let'

 let list=new LinkedList<String>();

The spec of C# 3.0 will use the same mecanism but with the kerword 'var' instead of 'let'.

 var list=new LinkedList<String>();

James Gosling describes in a blog entry another possible syntax, using ':=' instead. This syntax has the great advantage to not introduce a new keyword.

 list:=new LinkedList<String>();
 employeeList:=entityManager.findAll(Employee.class);

Using the james's syntax, the grammar of the JLS can be patch by adding a new production DRYVariableDeclaratorRest :

  BlockStatement :
        LocalVariableDeclarationStatement
        ClassOrInterfaceDeclaration
        [Identifier :] Statement

  LocalVariableDeclarationStatement:
        [final] Type VariableDeclarators ;

  VariableDeclarators:
        VariableDeclarator { , VariableDeclarator }

  VariableDeclaratorsRest:
        VariableDeclaratorRest { , VariableDeclarator }

  VariableDeclarator:
        Identifier (VariableDeclaratorRest | DRYVariableDeclaratorRest)

  VariableDeclaratorRest:
        {[]} [  =   VariableInitializer]

  DRYVariableDeclaratorRest:
        := VariableInitializer

You can verify that it is integrated smoothly with the other rule and that such a local variable can be declared final or can have annotations.

Update: december, 15ht, the following idea is stupid, sorry.

There is a small gotcha with the foreach loop. If one decide to just omit to declare the type like this:

 for(s:list) {
   System.out.println(s);
 }

the grammar will contain a case where an identifier can be a type or a variable (like in C++). To avoid that, i propose to introduce another colon to disambiguiate the grammar. Thus a code that declare a local variable in a foreach loop should be like this:

 for(s::list) {
   System.out.println(s);
 }

The grammar with a production that manages infered local variables.

ForVarControl:
        [final] [Annotations] ForVarDeclarator

ForVarDeclarator:
        Type Identifier ForVarControlRest
        Identifier :: Expression

ForVarControlRest:
        VariableDeclaratorsRest ;  [Expression]  ;  [ForUpdate]
        : Expression

I hope the next jdk will include more than a shorter instance creation syntax.
I wait your comment.
Rémi

java.lang.Unreachable as type argument

The closure proposal specifies a new type java.lang.Undeclarable that can be used as a return type of a method to indicates that this method never returns. All instructions after a call to a method that returns Undeclarable is unreachable, by example :

  Undeclarable throwAnIOException(String message) throws Exception {
    throw new IOException(message);
  }
  ...
  void foo() {
    ...
    throwAnIOException("oups");
    return; // this code is unreachable
  }

This type is needed because in the closure proposal, the closure (return) type is infered from the type its last expression. So if the last expression is a throw, the compiler will infer java.lang.Undeclarable.

Now, the proposal also provides an example of code in which Unreachable is used as argument of a type variable.

  interface NullaryFunction<T, throws E> {
    T invoke() throws E;
  }
  NullaryFunction<Unreachable,null> thrower = {=> throw new AssertionError(); };

But Undeclarable can only be used as a return type and the compiler will not enforce that.
I propose to tag the type variable with the keyword return to say that type variable can only be used as return type, with the following rules

• Its a compile error to use Undeclarable as a type variable not tagged by return.
• Its a compile time error to use a type variable tagged by return in any other places that as return type

With these rules, the previous example will be:

  interface NullaryFunction<return T, throws E> {
    T invoke() throws E;
  }
  NullaryFunction<Unreachable,null> thrower = {=> throw new AssertionError(); };

The closure proposal already describes a resticted type variable: the throws type variable. A type variable tagged by throws that can be only used in throws clause and used a special inference mecanism. So adding a return type variable is just another way to restrict a type variable.

In a recent blog entry, Peter ahé's describes Self types, he write in an answer to one of my comment, "The this type variable is only type-safe in covariant places". So like Undeclarable, if we want to use this as a type argument, the corresponding type variable should be tagged by return.

I wait you comments.
Rémi

How far is fidji - Reloaded

This blog is an infrared echo to Matthias Ernst's last post titled "How far is fidji".

1. get rid of checked exceptions :
   For me the problem is not checked/unchecked exception but the fact that in the JDK, some places abuse of checked exceptions like reflection, rmi etc. I agree with matthias that it could be easy to add a keyword to transform or not xhecked exception to runtime exception but a checked exception like IOException is vital for a decent programming language. You can't reuse a lib that performs networking and get rid of checked exceptions.
2. fill the void :
   One proposed annotation of the JSR 305 is about that.
3. add type inference for locals:
   I totally agree with matthias, you don't have to declare local variable if you won't. And i hope it will be included into dolphin. C# has already that with the keyword var.
4. add reliable closing of iterators in the for-each loop:
   I prefer to use a JDK method for doing that, by example, using the proposed for syntax by neal gafter.
5. Resource literals: The ressource section of the JSR 277 is not a mecanism close to what matthias want.

So I don't know how far we are from Fidji but i know that we are not so far from Java :)

Stupid question: Why the creation of array of parametrized type is unsafe ?

I don't understand why array creation of array of parametrized type is forbidden by the JLS.

Let me take an example :

  public class Holder<E> {
    public void set(E element) {
      this.element=element;
    }
    public E get() {
      return element;
    }
    private final E element;
  }
  ...
  Holder<String>[] holders=new Holder<String>[4];
  for(int i=0;i(); 

Why the array creation new Holder<String>[4] is rejected by all java compiler as unsafe ?

The JLS3 clearly doesn't allow that in section 10.3, about array creation "It is a compile-time error if the element type is not a reifiable type". Okay, the JLS prohibits that but i don't understand why. Is there a Java guru that can explain me why ?

The generics tutorial written by Gilad Bracha provides an example that show that there is a problem when an array is assigned to an Object. In short, the following code is unsafe :

  Object[] array=holders; 
  holders[0]=new Object();
  System.out.println(holder[0].get()); // try to call get on an Object

That's true, but i don't agree with the diagnostic. For me, it's the convertion to Object[] (Object[] array=holders) which is unsafe and not the array creation.

I wonder if the spec can't be change to something like this :

It's unsafe to convert an array of parametrized type to Object, Object[], Serializable and Cloneable.

This condition is far less restrictive and permits to use parametrized array in lot of usefull use cases. By example, the following code will be safe.

 public static void main(String[] args) throws Throwable {
   final InetAddress host=InetAddress.getByName("www.playboy.com");
    
   int poolSize=4;
   ExecutorService executor = 
     Executors.newFixedThreadPool(poolSize);
   Future<Long>[] futures=new Future<Long>[poolSize]; //ok
   for(int i=0;i<poolSize;i++) {
     futures[i]=executor.submit(new Callable<Long>() {
       public Long call() throws Exception {
         long time=System.nanoTime();
         host.isReachable(2000);
         return System.nanoTime()-time;
       }
     });
   }
   executor.shutdown();
   for(Future<Long> future:futures)
    System.out.println("reach "+host+" in "+future.get()+" ns");
 }

What do you think about that ?
Rémi

Reified generics in Java

Neal Gafter post a blog entry about adding reified generics to Java. I likethat proposal mostly because developpers will have the choice betweeen reified or not reified generics.

About the syntax, i think it's better to use an anotation than to re-use the keyword class. Annotating a type variable is not currently allowed but it seems that the upcoming JSR 308 will allow that.

 class ArrayList<@Reified E> {
   ...
 }

 class ArrayList<class E> {
   ...
 }

The big question is why tagging the type variable and not the parametrized type.
I'm not sure that allowing to declare on a same parametrized type refied type variable and non refied is a good idea, because i don't see a use case.

 class HashMap<@Reified K,V> {

 }

Here, HashMap is not refiable because V is not refiable. So what is the need of half reified type ?

Else, i not sure about the fact that allowing reified generics requires to change collection interfaces. Allowing reified interfaces will just allow safe cast/instanceof, i wonder if it worth all problems linked to the introduction of a new collection hierarchy. It's perhaps up to a collection implementor to decide if a collection implentation will use or not reified generics.

To end, i've recently read a paper of Mirko Viroli about how to represent wildcard types at compiler level but i'am not aware of any work about that at VM level.

Rémi

JSR 277 and ahead of time compilation

Last week, an early draft of Java Module System (JSR277) was published. To sum up, this draft defines :

1. the concept of exportable module, which is closed to OSGi bundle or eclipse plugin.
2. the way modules are packaged, as super jar named Java Archive Module (or jam).
3. and how modules are store in a local repository.

One short paragraph at the end of the section 12 of the chapter 7, the one about module repository, awakes my geek instinct. I quote:

 This deployment step offers the repository implementation an
 opportunity to transform a module definition into a more efficient
 format for run time access. For instance,
 [...]
 Native code can be generated for the frequently used classes in a
 module definition to optimize runtime performance.

Let me try to explain what is the idea behind : when a module is stored in the local repository, the code is automatically transformed in native code by performing a static analysis on the bytecode. This technique is called Ahead of Time compilation and is, at least, used today by Excelsior JET runtime environment. Then when the VM try to execute a class of this module, the native code is used for the first runs before the hotspot detection that performs a runtime analysis provide a better code.

Oh, oh, will we see in a near future, more VMs providing a module repository implementing ahead of time compilation ? And thus VMs that never interpret bytecodes !

Rémi Forax

Obfuscated Java

With a colleague, we discuss about the fact that function type can or not obfuscate Java code and he advocate the fact that a code in Java is always readable if you have an IDE that can auto-format the code.
The following example doesn't answer to the question about function type but shows that it is always possible to write an obfuscated code even in Java. This example is extracted from the BlueJ mailing list.

/*   Just Java
     Peter van der Linden
     April 1, 1996.

\u0050\u0076\u0064\u004c\u0020\u0031\u0020\u0041\u0070\u0072\u0039\u0036
 \u002a\u002f\u0020\u0063\u006c\u0061\u0073\u0073\u0020\u0068\u0020\u007b
  \u0020\u0020\u0070\u0075\u0062\u006c\u0069\u0063\u0020\u0020\u0020\u0020
   \u0073\u0074\u0061\u0074\u0069\u0063\u0020\u0020\u0076\u006f\u0069\u0064
    \u006d\u0061\u0069\u006e\u0028\u0020\u0053\u0074\u0072\u0069\u006e\u0067
     \u005b\u005d\u0061\u0029\u0020\u007b\u0053\u0079\u0073\u0074\u0065\u006d
      \u002e\u006f\u0075\u0074\u002e\u0070\u0072\u0069\u006e\u0074\u006c\u006e
       \u0028\u0022\u0048\u0069\u0021\u0022\u0029\u003b\u007d\u007d\u002f\u002a

 */

For those that have problems with reading unicode characters encoded in hexa :)
It's equivalent to

class h {
  public static void main(String[] args) {
    System.out.println("Hi!");
  }
}

Rémi Forax

Languages Evolution: introduction of new keywords

When you want to add features to a language without breaking backward compatibility, a widespread idea that you can't add new keywords.

That is why we can currently see weird proposal in Java space that try to reuse old keywords to express new kind of abstraction, by example, synchronized (closure v0.2 section 3) or (Neal Gafter blog about for).

When you specify a new keyword, you need to change the lexer to recognize sequence of characters as a new token. Thus the lexer doesn't recognize this sequence as an identifier anymore.

One magic solution is to use a special character (or more) for differenciate keyword from identifier. Lot of scripting languages use '$', '#' etc. to tag variables, Perl6 is the best example.
Scripting language use special caracters not only to simplify the lexing process but to help their runtime system to choose between overloaded operations. So adding a new keyword is not a major problem for those languages.

Java is a strong typed language so it doesn't need such special characters and we are stuck while we continue to see lexers as lex. The problem comes from the lexer, so the solution is to change how lexer works.

Let me take an example, "enum" is a new keyword introduced in 1.5 to declare enumerated type. So the lexer of an 1.5 compiler now recognize "enum" as a keyword in the whole program.
But in fact, the "enum" that interests a language designer is only needed to recognize "enum" as a keyword in the case of a type declaration not in a block of code.

The solution is to use a lexer that implements contextual keywords, i.e a lexer that let the parser activate or not rules needed to recognize tokens depending on the parser state.

enum Foo {                                 // keyword
  public static void main(String[]) {
    Enumeration enum=... // identifier
  }
}

With two colleagues, i've written a new Parser Generator named Tatoo that generates this kind of lexer.
The tutorial is in french at this time because we haven't lot of time and by our students need it. But a translation will be available soon. Slides in PDF and an article from PPPJ'06 are available in english.

Tatoo contains other innovative features like grammar versioning, full NIO support (push lexer/parser), lexing without unicode decoding, AST generator. I will blog about those features later.

About iterative control abstraction

Neal proposes to use for to tag methods that take a synchronous closure as parameter and to call this new kind of method.

It's better than to use synchronized as proposed before but (there is always a but :) i see two drawbacks.
First, for is used to tag the whole method and not one of its parameter so there is no way to define a method that takes a synchronous closure and an asynchronous one. Second, in Java, for is an instruction and not an expression but synchronous method may return a value.

  int sum=for each(int sum,Integer value:Arrays.asList(2,3),0) {
    sum+value
  }
  ...

Else, there is a bug in signature of the method eachEntry proposed by Neal. He uses the syntax of the closure v0.2 and as i was explained here if you don't use function type, you have to care about subtyping relationship between parametrized types.
The code above is not legal with the signature given by Neal void for eachEntry(Map<K,V> map, Block2<K,V,E> block) throws E.

 Map<String,Droid> map=...
 for eachEntry(CharSequence droidName, Droid droid : map) {
   ...
 }

The type of droidName must be the same than the first type argument of the map, here String and not CharSequence.
The method signature eachEntry must be changed to :

public interface Block2<K,V,throws E> {
    void invoke(K k, V v) throws E;
}
public static <K,V,throws E>
void for eachEntry(Map<K,V> map, Block2<? super K,? super V,E> block) throws E {
    for (Map.Entry<K,V> entry : map.entrySet()) {
        block.invoke(entry.getKey(), entry.getValue());
    }
}

I still prefer the closure v0.1, i think dispite the fact that it introduce a new type (function type), it's simpler to read.
The method each of my first example with the v0.2 is coded like this:

  public interface Expr1<V,R> {
    R invoke(V v,R r);
  }
  public static <R,V>
  R for each(Collection<? extends V> values,R initialValue,
   Expr1<R,? super V> expr) {
    for(V value:values)
      initialValue=expr1.invoke(value,initialValue);
    return initialValue;
  }

  public static <R,V>
  R each(Collection<V> values,R initialValue,R(V v,R R) expr) {
    for(V value:values)
      initialValue=expr1(value,initialValue);
    return initialValue;
  }

Rémi

Use foreach to iterate using an Iterator

When the foreach syntax (for(:)) has been introduced in 1.5, a recurring question was why foreach is not able to iterate using an iterator.

I think i have a trick to do that using the syntax of the closure :

 Iterator<String> scanner=new Scanner(System.in);
 for(String word:() { scanner })
   System.out.println(word);

Why this code works. The foreach statement needs an array or an object that implements java.lang.Iterable. Iterable is an interface that has a single method iterator that returns an iterator thus the closure convertion is applicable.

Rémi

Is function type required for closure ?

As seen in Neil Gafter's blogs, there are two rival proposals about adding closure to Java language. The first one, named v0.1, introduces a new kind of type, function type in order to express the type of a closure. The second one, named v0.2, enables to use interface as type of closure.
The goal of this entry is to show the pro and cons of the two proposals.

Let me take an example. I want to add each integer of a list, so i can use a closure for that. The code bellow uses the abbreviated invocation syntax of closure defined in section 10 of the two proposals.

 List<Integer> list=Arrays.asList(2,4,6);
 int result=0;
 forEach(int value:list) {
   result+=value;
 }
 System.out.println(result);

The code that declares and uses closure is the same for the two proposals. A contrario, the code that defines the method forEach is different depending of the version of the proposal.
With the proposal v0.2, closures are typed by an interface, like that:

 public interface Performer<E> {
   void doIt(E element);
 }
 public static <T> void forEach(
   Iterable<? extends T> iterable,Performer<? super T> performer) {

   for(T item:iterable)
     performer.doIt(item);
 }

The main interest of this way is the seamless integration with existing codes. The code of forEach is a "traditionnal" 1.5 code with no supplementary syntax. Futhermore, using an interface permits to name abstraction, it seems more clear to talk about a comparator than a int(T,T).

Now, the same example but with the syntax describes in the v0.1.

 public static <T> void forEach(
   Iterable<? extends T> iterable,void(T) func) {

   for(T item:iterable)
     func(item);
 }

Clearly, you have to learn a new syntax: void(T) is a function that take a T and returns nothing. Apart that fact, the syntax is not hard on the eyes like, by example, function pointer in C.

The subtyping rules defined between function types avoid to understand generics deeply, void(T) seems more simpler than Performer<? super T>.

In order to use closures with an existing code the v0.1 includes the same convertion mecanism between closure and interface that the v0.2. So you have two ways to define forEach, not a good point !

The bytecode used for invocation is different, the v0.1 will use invokedynamic and the v0.2 invokeinterface. The difference is that invokedynamic don't need to box and unbox, so the performance could be far better using a function type than using an interface. The cost of boxing could really be a problem because in general closure are small code and the cost of boxing will not be negligible comparing to the cost of executing the instructions of the closure. Perhaps, it's not a big beal, Java has lives with that cost since its inception and it doen't seem to be a big trouble.

And i want to end with a stupid argument, the proposal of the v0.1 takes less line of codes that the v0.2 :)

Now express yourself, what do you think ?

Synchronized or confined

A new version of the closure proposal has been post at the end of this week by Neal Gafter and proposes in section 3 to tag parameter with a special keyword to differenciate between synchonous and asychronous closure.

For me this section contains two flaws, the first one is that the keyword used is synchronized, the second one is to not recognize that such feature can be usefull in other contexts than closure.

In the Java API, there are existing methods that take an object as argument and don't allow to store this object in a field, by example, startElement/endElement of SAX handler or RowFilter.include().
I think a unique keyword can be used in all this cases.

Now, why not using synchronized for that ?
synchronized means thread, threading model, memory model, etc. This keyword is so important that it can't be reused to express something totally different without complexified something that is already not obvious.
I think that using an annotation in this case is better. You can name it as you want, there is already a reflexion support. I propose @Confined because the reference can't escape.

The following code shows the usage of @Confined with closure :

  public static <T> void forEach(Iterable<? extends T> iterable,@Confined void(T) func) {
    for(T item:iterable)
      func(item);
  }
  ...
  List<Integer> list=Arrays.asList(2,4,6);
  int result=0;
  forEach(list,(int value){
    result+=value;
  });
  System.out.println(result);

Rémi

With one breath, with one flow
You will know
Synchronicity
                             --- Synchronicity / Police

Using jrunscript as a build tool

In my last post, i've described how to use jrunscript to create a build script.
As olivier wrote in the comments, the build process is usually a dependency graph and jrunscript doesn't do that by default.
But javascript is far more powerful than XML thus it's easy to create commands that enable to declare dependencies betweeen functions as Ant or make does.

I propose the following commands:

1. depends_on that declares a set of dependencies of a function.
2. run_targets that explores the graph from a function in a dependency first order.

Javascript enables to put data in an existing object even a one like Object or Function. In fact, objects are considered as an hashtable of couples property/value so the dependencies of a function can be stored in the function itself.

The build script:

function test1() {
  echo("test 1");
}

function test2() {
  echo("test 2");
}
test2.depends_on(test1);

function test3() {