Cay Horstmann's Blog

Dr. Gafter comes to SJSU

I teach a graduate programming languages class at San Jose State University. In order to inject some topics of current interest, I had a lab about closures and the competing closure proposals for Java 7. I got an email from Neal Gafter: “Hey, it's really cool to see your reference to BGGA in a SJSU lab assignment!” I asked if he could give a talk at the department seminar, to which he graciously agreed. We had a packed room today. I am very excited that my students had a chance to witness a bit of history in the making. Here are my impressions of the talk.

The End of History - Not.

I have taught CS252 for many years. I cover the lambda calculus, closures, continuations, metaprogramming, and other advanced programming language topics. For several years, my graduate students viewed the material as having little relevance to their lives. Clearly, real programming languages had evolved from C to C++ to Java, and the end of history had been reached. Every other language was either some poorly designed pile of crud or an ivory tower obsession.

I tried what I could, talking about continuations for web applications and Paul Graham's article on Lisp in a startup. My students, however, are by and large a pragmatic bunch. They know that web applications aren't written in Lisp, and googling for Paul Graham led them to this funny but unflattering article.

This year, things were finally going my way. The buzz behind Ruby and Rails is hard to ignore, and it seems pretty clear that Java will get closures in some form or other. Learning about metaprogramming and closures doesn't seem a waste of time any more.

How Not to Write a Proposal

Alex Miller has a nice blog on Java 7 proposals, including the competing closure proposals. My students found the CICE proposal pretty straightforward. It starts out with a simple promise:

    new Thread(new Runnable() {
        public void run() {
            foo();
        }
    }).start();

 new Thread(Runnable(){ foo(); }).start();

The first impression is that this is easy and convenient.

In contrast, the BGGA proposal starts out with:

I translated this into plain English in my lecture:

The proposal then goes on to discuss closure literals, function types, closure conversions, exception type parameters, definite assignment, the type Unreachable and control invocation syntax. That section finally contains a compelling example:

with(FileReader in : makeReader()) with(FileWriter out : makeWriter()) {
    // code using in and out
}

No more try/finally/in.close. Hooray!

When we discovered that BGGA gives us this and CICE doesn't, we were getting much more excited about it.

The Presentation

Neal told me before the talk that he has given this presentation fifteen times already—someone ought to make a movie about him tirelessly criss-crossing the planet and spreading the message :-) Actually, you can see a video of the presentation here.

He started out with this quote:

"In another thirty years people will laugh at anyone who tries to invent a language without closures, just as they'll laugh now at anyone who tries to invent a language without recursion." - Mark Jason Dominus

I am not sure whether my students understood this—they had probably never seen a language without recursion. Sure, I tell them in the undergraduate programming languages class about Fortran, but they view it as a freakish thing, like a two-headed cow, not as something that was taken seriously in its time. But Neal went on to say that one now says the same thing about garbage collection, and my students do remember the dark ages of C++ and having to call delete (but not too often).

Neal systematically defined closures, discussed why anonymous instance creation expressions fell short, defined the requirements for a true solution, went over syntax, and finally presented the examples. The talk is much easier to follow than the spec, and everyone enjoyed it tremendously.

Still, if I had given the talk, I would have given the examples first. Since this is my blog, I get to do it my way, so here goes.

Closing Locks, Files, Database Connections

Don't you hate writing this code?

Connection conn = dataSource.getConnection();
try {
   . . .
} finally {
   try {
      conn.close();   
   } catch (SQLException ex) {}
}

On a bad day, I even worry about a SQLException that triggers the finally clause, only to have a second exception occur in conn.close() that masks the first one.

With BGGA closures, you write

with (Connection conn : dataSource.getConnection()) {
   . . .
}

Iterating over Maps

I can't ever remember whether it is

for (Map<String, Integer>.Entry entry : map.entrySet()) {
   String key = entry.getKey();
   Integer value = entry.getValue();
   . . .
}

or

for (Map.Entry<String, Integer> entry : map.entrySet())

I either look it up in Core Java or, if I am lazy, I sheepishly fall back on

for (String key : map.keySet())

and feel bad about paying for the additional get.

With BGGA, I can have

for eachEntry(String key, Integer value : map) {
   . . .
}

I want that.

For that matter, I want to iterate over a two-dimensional array as

for eachIndex(int i, int j : matrix)

without abominations such as a[i].length. I also want to have both index and value when iterating over a list. With BGGA, I can make control structures for those purposes.

Callbacks

All the closure proposals handle this use case, but here goes, for the sake of completeness:

myButton.addActionListener({ ActionEvent event => model.myButtonAction(); });

It seems a shame that one has to do something with the useless ActionEvent. Ideally, I'd like to be able to ignore the parameter and write

myButton.addActionListener({ => model.myButtonAction(); });

But maybe it's asking too much.

Logging Exceptions

Neal didn't talk about this, but I gather from the spec that this must be possible.

I hate writing reflection code like this:

try {
   m.invoke(arg1, arg2);
}
catch (IllegalAccessException ex) {
   logger.log(Level.SEVERE, "", ex); 
} catch (IllegalArgumentException ex) {
   logger.log(Level.SEVERE, "", ex); 
} catch (InvocationTargetException ex) {
   logger.log(Level.SEVERE, "", ex); 
} 

If I understand the proposal correctly, I can instead write

<throws IllegalAccessException|IllegalArgumentException
      |InvocationTargetException>.tryAndLog(logger) {
   m.invoke(arg1, arg2);
}

It's still not pretty, but I can't fault BGGA for a cumbersome API with a method that throws three unrelated exceptions.

Process and Politics

Neal talked a bit about the process of getting a language extension accepted for the Java language. The JSR process seems rather company-centric and doesn't seem to deal all that well with the possibility that two people from the same company (in this case, Neal Gafter and Josh Bloch, both of Google) have differing views. Still, he hinted that something will happen in the near future about getting a JSR on closures started.

Overall, my impression was that Neal's approach is right. You want to be able to take a block of code, wrap it up and hand it to some other code and just have it work. Anonymous inner classes won't do it, no matter how much syntactic sugar you sprinkle over them. When evaluating the competing proposals, I suggest that you look at the use cases first before you get flummoxed by the syntax (which may well change) and the arcane considerations of definite assignment.

Comments

• May I steal your text (with attribution) and post it as part of a brief tutorial?

  Posted by: gafter on April 12, 2007 at 11:37 PM

• Note that CICE is designed to be used in conjunction with ARM blocks. ARM blocks provide a simple, straightforward solution to the try/finally/in.close problem. In fact it's arguably better than any solution that you could build atop BGGA closures, as it can handle multiple resources without multiple nested blocks. (This is a common complaint against C#'s "using" statement..)

  Regards,

  Josh

  Posted by: jbloch on April 13, 2007 at 12:50 AM

• Josh, you have my vote!

  CICE solves the problem. BGGA is a wet dream for people that loves coding (and functional programming) but a nightmare to read for Java Joes.

  Sorry for ruining wet dreams, but most developers are Java Joes. Java is made for Java Joes which is why it is so successful. If you don't want to code in a language that is simple to write and read I suggest Groovy. It packs a lot of power but is still reasonably easy to read.

  Cheers,
  Mikael Grev

  Posted by: mikaelgrev on April 13, 2007 at 02:37 AM

• Cay, you forget to mention that the CICE example is a solution that is doable out of the box. The BGGA examples is only doable if you ahve also added those closure definitions. This means that with the BGGA proposal will in effect create you own "sub language" with very powerful syntaxes and semantics. And that would be a good thing?

  Posted by: mikaelgrev on April 13, 2007 at 02:48 AM

• Its unfortunate that your comparison of competing closure proposals only seems to have compared those from Silicon Valley ;-)

  The FCM proposal - http://docs.google.com/Doc?id=ddhp95vd_0f7mcns - covers the same use cases as BGGA - you can manage a resource, loop over maps or lists with an index etc. You can also add callbacks too.

  FCM and BGGA differ in the detail. What should return do within a closure block? How should a result be returned back to the invoker? Is it acceptable for the application to receive UnmatchedNonLocalTransferException? Should the change try to blend with the rest of Java, and the expectations of Java developers?

  As a summary, I would say that CICE aims to solve two or three specific problems - closures (BGGA or FCM) aim to solve the generic problem.
  

  Posted by: scolebourne on April 13, 2007 at 05:35 AM

• Stephen,
  I though FCM and CICE were more alike, but maybe you just have hidden the complexities in a nicer, more Java-like, syntax than BGGA?

  It would be nice if someone without much bias posted something about the differences between FCS, BGGA and CICE...

  Syntax wise I think both FCS and CICE is on the right track. As long as you can't define something that look like meta-language I'm happy. A function should look like a function and a keyword should look like a keyword IMO.

  Posted by: mikaelgrev on April 13, 2007 at 06:17 AM

• this ARM business looks fairly similiar to pythons/jythonx's context managers. An object defines an __enter__ and __exit__ methods that get invoked upon entry of a block and the exit of a block. Of course a difference appears that __enter__ returns an object that you can actually use. __exit__ is invoked with some arguments if an exception is thrown.

  leouser

  Posted by: leouser on April 13, 2007 at 06:47 AM

• @Neal: By all means, steal what you find useful.

  @Josh: Thanks for letting me know about ARM. Do you want to give a colloquium talk?

  @ Stephen: My apologies--like Mikael, I thought that FCM was somewhat like CICE. I'll have another look.

  @Mikael: I am not sure I understand the issue about complicated syntax in BGGA. The "with" and the various "for each" examples look harmless to me. The listener example looks just like Groovy, with a => instead of a ->.
  

  Posted by: cayhorstmann on April 13, 2007 at 07:33 AM

• FCM consists of two parts - FCM and JCA. The FCM part came first and adds proposes a number of things including inner methods. An inner method is close to BGGA in terms of minimal syntax (no reference of the class, such as ActionListsner), but close to CICE in terms of the behaviour of return. In fact FCMs closest cousin is probably Groovy.

  The JCA part of FCM is new, and allows the withLock / withFile type of closure keywords. Here return will return the enclosing method.

  I did a (reasonably) unbiased comparison at http://jroller.com/page/scolebourne?entry=java_7_comparing_closure_proposals and http://jroller.com/page/scolebourne?entry=comparing_closures_2_more_examples (which predates JCA).
  

  Posted by: scolebourne on April 13, 2007 at 08:30 AM

• Howard Lovatt, the C3S proposal author did a nice, and elaborate comparison of the 4 major closure proposals at : http://artima.com/weblogs/viewpost.jsp?thread=202004
  I am hoping that any closure proposal that ends up being accepted will be the best of breeds of all these competing ones. As for my own wish I would really like to see FCM's method literals (or currying in general) being adopted as well in one form or another.

  Posted by: mikeazzi on April 13, 2007 at 08:57 AM

• ARM looks like a hardcoded hack that you get for free with BGGA (or JCA).

  Posted by: tompalmer on April 13, 2007 at 09:52 AM

• @Cay: I'd be happy to give a colloquium when time permits.

  @TomPalmer: You say "ARM looks like a hardcoded hack that you get for free with BGGA (or JCA)." But one man's hardcoded hack is another man's purpose-built control construct. If it's purpose built, you can make it better (e.g. you can support multiple resources in one block), and its behavior is like all of Java's current control constructs(which aren't based on closures). Finally I would argue that you don't get anything "for free" with BGGA; you get it at the cost of great complexity in an area where Java is already suffering from excess complexity (the type system).
  

  Posted by: jbloch on April 13, 2007 at 10:05 AM

• @jbloch: I'm a closures person, but thanks for the rebuttal. It's good to get the topics on the table.

  Posted by: tompalmer on April 13, 2007 at 10:46 AM

• Cay, the usage of the closures in BGGA looks and are simple. It is understanding the signature of them, what they actually do regarding return statements and which variables they modify. Also writing them that is complicated and easy to get wrong.They look and feel like normal keywords, which is negative in my book.

  Posted by: mikaelgrev on April 13, 2007 at 02:30 PM

• 
  @Michael, I understand what you mean about complex signatures. In that regard, aren't closures similar to generic methods? The signature static <T> void sort(List<T> list, Comparator<? super T> c) doesn't look pretty. But so what? A few expert programmers create these utility methods and lots of programmers use them all the time without worrying about the gory details.

  
  As for what variables they modify and what return means, I think that's where BGGA closures shines. It always means what it would mean if the code had been a block in the defining scope. At least that's how Neal pitched them in his talk. Is there some nastiness that I should know?
  

  Posted by: cayhorstmann on April 13, 2007 at 03:01 PM

• I think that wildcards are too hard, and I think it should be easy for everyday programmers to write usable APIs. And (ignoring wildcards - and I'd love to get rid of them despite the static-typing wonders) I think closures APIs are easy to create and use. In languages that have closures, people use them all the time without batting an eye. For common use cases, they really aren't hard at all.

  And not that I'm anyone important (because I'm not), but I think that FCM with JCA is the best proposal right now in terms of being obvious about how to use them and behaving in expected fashions.

  Posted by: tompalmer on April 13, 2007 at 03:40 PM

• Cay, you asked what nastiness you are missing. The problem is that return actually makes more sense returning to the invoker in most circumstances (callbacks and methods like sort) - as in FCM.

  BGGA uses return exclusively for returning to the enclosing method. This means that there has to be a separate syntax for returning to the invoker of the closure. In BGGA at present this is last-line-no-semicolon, see http://www.jroller.com/page/scolebourne?entry=comparing_closures_2_more_examples. As example 3 from this link tries to demonstrate, last-line-no-semicolon results in ugly code.

  Put simply, BGGA states that having one semantic for return in all types of closures is the best solution. Whereas FCM-JCA says that return should act like an inner class return when the closure is used like an inner class (FCM), but act like an enclosing method return when the closure is used like a keyword (JCA). Our natural reaction as developers is to choose the single consistent solution to any problem - BGGA in this case. Its only when you get to try it for real that the FCM-JCA approach shines through as more usable within the context of Java, and the expectations of existing Java developers.

  Its actually a pity there's no 'closure's shootout' at JavaOne really ;-)
  

  Posted by: scolebourne on April 13, 2007 at 04:49 PM

• Cay, the benefits of generics vastly outweighs the benefit of closures and therefor the extra complexity is IMO worth it.

  Regarding return nasties, check out paragraph 7 of this page: http://www.artima.com/weblogs/viewpost.jsp?thread=202004

  When you say that a few expert programmers should create the closures I think a reference to male driving is in order. About 80% of male drivers think they are above average, or so a survey I read a few years ago said. So, should we have some kind of test that make a developer "closure creator ready"?

  Seriously, closures are syntactic sugar that makes sense if you create a new language. It is harder to squeeze in to an already established language in a clean way. If terribly important we can lift the "final" restriction on inner classes and educate people about the "OuterClass.this" notion that some seem to have forgotten. Maybe even having some sugar as CICE proposes, but that's it.

  FCS has a really nice method/field/class literals think going and that could be extended to a property literals, which would be extremely useful and really make things easier. Take for instance JSF properties which nothing but an abomination. Those could be one-liners if properties was introduced...

  Posted by: mikaelgrev on April 14, 2007 at 12:56 AM

• Agree with Mikael Grev. My vote too goes for Josh.
  
  - Bharath
  

  Posted by: bharathch on April 14, 2007 at 01:10 AM

• Cay, also look at paragraph 11 of the paper I linked to in my earlier post as it is quite clearly explains the problems with the different types of returns in BGGA.

  Posted by: mikaelgrev on April 14, 2007 at 01:18 AM

• Concerning method calls that look like keywords, IDE-less Ruby users don't seem to get confused over this. And Java users usually have IDEs. If syntax highlighting (and your own memory) doesn't work for you, just mouse over, and the IDE can tell you what kind of word that is.
  Concerning resource closing, I find nested blocks of allocation (like C# or Ruby) easier to make sense of than C++ or reference-count-oriented Python's use of RAII. So, while ARM (and the sample in the BGGA spec) aren't quite as freeform as RAII, I tend to lean towards one resource per block. For readability purposes.

  Posted by: tompalmer on April 16, 2007 at 02:29 PM

• Well, multiple resource allocations per block could be handy if you want the callee to determine order for you (such as with Neal Gafter's nice withLocks example), but otherwise I like to see one resource per block.

  Posted by: tompalmer on April 16, 2007 at 03:27 PM

This work is licensed under a Creative Commons License.