Kirill Grouchnikov's Blog

Conspiracy theory about closures in Dolphin

I was watching the (not so impressive for now) Flying Irish on Saturday and my mind wandered off. Neal Gafter's original proposal for introducing closures into Dolphin along with code samples and what not has been published only two weeks ago and already the blogosphere is abuzz (more on that later). And i started to think - it's like a crime movie and all those cheap police thrillers. When you try to solve a murder, think about who's the first to profit from that death and you have your killer. Now, who's going to profit from having closures in Java and all those corner cases? Of course, the writers of "Java puzzlers"!!! Hey, isn't Neal one of those? Of course!!! Isn't Neal behind the closure proposal? Of course!!!

And now on a more serious note. We have tens of discussions (notably this Javalobby thread initiated by Mikael Grev, this SwingLabs thread and Remi blog right here on java.net). We have scores of proposed syntax changes and hundreds of opinions. Somebody on one of those threads (don't have a link at hand) said "People either understand closures, or don't like it". Well, i understand closures but i can't say that i like them. I feel that it's too disruptive for a such widespread language (and i completely back up Mikael's original post on Javalobby). Does it make programs easier to read? In some cases - yes. Does it bring more clarity to programs? In some cases - yes. Is it easily abused by inexperienced programmers, novices and showoffs? Almost certainly. Does it add yet another way to do what can be done already with existing language constructs (this is what makes Swing so hard for the beginners)? Certainly. Can it make harder to join an existing project and fix somebody else's code? Almost certainly.

Now, language changes in Tiger were both a blessing and a curse. Generics offload some of the error screening to the compiler stage, while making code look more complicated. Autoboxing / unboxing make working with primitive types easier, while introducing hard-to-track bugs, especially in collections of primitive values (see "Java puzzlers" session from JavaOne 2006). Enhanced for-loop is less handy than i originally thought since it doesn't provide a way to remove / index entries. And the list goes on. But, and this is a major but. All these changes were minor compared to what is planned for Dolphin. In Dolphin, we are talking about inferred types, function pointers, omitting entire language constructs and so on. It's not eighties - most of this code is generated in IDE when you press Ctrl+Space, guys. And it's not a problem to read it as well.

Be that as it may, the widespread coverage of the proposed syntax is excellent for both the proposal and for the community. The only thing that makes me wonder is the seemingly selective choice of the community for this particular feature.

More than a year ago, Mark Reinhold outlined the proposed changes to Dolphin at JavaOne 2005. One of the major changes (no closures being mentioned at that time) was native XML support in the language. The community response was anemic at best. This entry on my own blog wasn't very popular (well, it may as well be my fault :), and the only other worthy entry was on Val's blog (which is kind of dormant lately). Sun's own XML engineers were politely shooshed during the Q&A section, and their opinion is nowhere to be seen on the web (and remember, that was more than a year ago). This year's JavaOne featured 158-slide presentation (downloadable here, session number 3441), and Mark followed up on his blog with only 3 (!!!) total comments. The server people seem to not care (well, most of the commercial applications still run on 1.4.2, so we're talking at least 7-8 more years until this gets to the application servers). In the meanwhile, the proposed features seem well-matured, and i'm sure that somebody at Sun already has working prototypes ready to go into Dolphin monthly builds.

Why everybody is such an expert at closures but not at XML? The potential for hard-to-read code is even greater with native XML support, yet nobody seems to care. If this is true (that nobody cares), perhaps we don't need it in the language at all... And i wonder yet again - why we don't hear about this from Sun's XML engineers (JAXWS / JAXB / FI / ... groups).

Comments

• With XML, I can close my eyes and hope that the notion goes away. With closures, it's very hard to look at the sponsors and think there is any real chance of it being rejected.

  Posted by: coxcu on September 04, 2006 at 07:35 PM

• Well, Mark is the chief engineer of Java SE if we're speaking about the sponsors...

  Posted by: kirillcool on September 04, 2006 at 07:49 PM

• well. i think making xml native to java does not hurt so much as long as they make it easy to use and difficult to abuse. it would make a lot of things easier, and posibly faster (xml-object transformation, parsing etc). But i dont know who would need it except the configuration happy JEE guys.

  Posted by: ahmetaa on September 04, 2006 at 08:20 PM

• I think we should avoid them both like the plague. Java is quickly blossoming into bloat ware. But maybe it's just me. Maybe my mantra of "less is more" is simply outdated in the age of GB RAM, GHz processors, and TB storage.

  It seems Sun has learned nothing from the RPC madness that gave us Corba which incidentally nobody uses but everybody is forced to drag around. Or the more recent WS-* stack which is a still born that we must all carry around, stinking up the place.

  Why don't they create a Java research branch where things like closures, generics, autoboxing, XML integration into the language, static imports, etc can be aired separate and apart from regular J2SE release. If there is massive uptake for the stuff going on in the research branch then phase it into J2SE gently. If it can't be gently phased in then create J3SE which doesn't have to be backwards compatible w/ J2SE. Because if you are going to do something radical don't do it half ass (i.e. generics) do right.

  Posted by: dfoster on September 04, 2006 at 08:37 PM

• You're right, maybe I should have been more vocal on XML in the Java language, but mostly I am almost ready to give up since the programming community seems hell-bent on locking onto schemas as the only way to do XML. I think your earlier blog that you referred to has some interesting ideas.

  Posted by: tobega on September 05, 2006 at 02:14 AM

• Well, in my case it was most likely because I'm ignoring everything that contains "XML". ;)
  I don't see any value of adding XML to Java and, well, I'm a bit shocked that that XML-thing is actually considered. What use should it be?
  Probably nobody discusses it because nobody beliefs this could really be considered. ;)

  Posted by: pago on September 05, 2006 at 03:07 AM

• A few years ago i was working for a big retailer, who was aggressively adopting Java, and having to upgrade hundreds of Enterprise servers, and was buying thousands Java Stations (which they never used and were subsequently donated to schools, in favour of using Linux PCs as "clients"). At the time, i cynically thought, Maybe this Java technology requiring so much RAM and CPU cycles is a conspiracy by a hardware company, which sells big iron with one hand, and pushes Java with the other ;) Actually i think it's a pity that Sun haven't managed to profit more from Java, because they deserve to. The only other company that has innovated more, in my opinion, is that granddaddy, IBM.
  

  Posted by: evanx on September 05, 2006 at 03:11 AM

• If my vote counts, I would say NO to XML and NO to closures. I like closures, but it is simply too late. For closures to be effective, all the core APIs should use them (e.g. Ruby).

  Posted by: vhi on September 05, 2006 at 06:16 AM

• I've spoken up against turning Java into just another XML dialect a year ago and again whenever it's mentioned.
  I don't want to write Java in an XML syntax, never mind that it's easier for tools to write and parse it that way (maybe, just maybe). Tools are supposed to make my life easier, not the other way around (and indeed most things in software engineering these days seem designed to make the software do less and the programmer more, annotations being one example).
  Pretty much the same with closures (which are just function pointers).If I want function pointers I'll use C and get bitten by trying to address illegal memory locations. I don't want the same in Java, and every idea I've seen about them so far does nothing at all except make very simple syntax much more complex (but possibly easier to generate for a code generator).
  I've already just about decided that I will NOT use any version past Java 5 given the crap introduced in Mustang and now proposed for Dolphin (the entire idiot idea about open sourcing the language specs being just the final straw), and may work to move my career away from what I'm ever more starting to perceive as a sinking ship now that Sun seems to want to turn Java into a mix of every other language out there that someone hypes in some publication, incorporating the worst of everything into the platform.

  Posted by: jwenting on September 05, 2006 at 06:43 AM

• I think most people recognised the XML proposal as being pure fruitcake.
  

  Posted by: tackline on September 05, 2006 at 07:01 AM

• Hmm, looking closer the XML stuff is not so bad... Need to get a nicer match syntax, though.

  Posted by: tobega on September 05, 2006 at 07:08 AM

• I've always been interested in closers, but never in XML. In my opinion, XML is a "compatibility layer", it bridges to other technologies for which there is no other means of communication. It’s used as the lowest common denominator and feels that way when you use it. When I get a file from a source that I can’t control I appreciate if it’s in XML with a well defined schema. When it’s applied other things I always seem to look at it and say, "That’s WAY more complicated then if I had written the code by hand". As far as closers go, I think they do have a place in the language. I was undecided until I read the article about closers and threads. It seems like they should just go together.

  Posted by: aberrant on September 05, 2006 at 07:15 AM

• 
  I'm with aberrant. As in, yes to closures. No to XML.

  Speaking more from my own perspective, if closures means easier to write and use anonymous inner classes, I'm all for it. I've seen lots of buggy code and hard-to-read code that would have been much better if closures were easier. And I'm tired of seeing such buggy and hard-to-read code.

  However, I've more than once written against XML in the language. And I still think it's a bad idea. Java already has a data model. And it's not XML. The most I'd really like to see here is better support for object literals (perhaps not unlike the object literals already supported in annotations). But I'd be okay without it, too. But definitely no to XML.

  Side comment: While not scientific and not exactly comparable, I still find these two poll results interesting: XML vs. closures.

  Posted by: tompalmer on September 05, 2006 at 09:33 AM

• Java is already too hard for the beginner. Adding closures and native XML won't really make it all that much worse, unless they use up a lot of new reserved words.

  I want to say that somebody has done this already, but if there's worry about beginners then they should have a compiler "-source" option for something like "1.7Lite" where a reduced feature/language set is enforced. IDEs would need to notice this flag and not provide auto-complete/syntax-highlight, etc. But it could be reversed and provide a "1.7Deluxe" and have closures/XML-native/etc. reside there instead of in the regular set. Just brainstorming...

  Posted by: gerryg on September 05, 2006 at 10:02 AM

• Everybody has their favorite feature which they want to add. I guess the web guys want XML and the computer scientists want closures. I could really use multiarrays and native complex number support -- something which has been in computer languages for over 40 years. Unfortunately the web guys are the majority (or have the money) and the computer scientists are in control.
  

  Posted by: vrob on September 05, 2006 at 10:53 AM

• vrob,
  That's the thing - web guys say nothing about XML. There's no joyous excitement in the web world about this, so i guess it doesn't mean anything to them.

  Posted by: kirillcool on September 05, 2006 at 10:59 AM

• Kiril--it's no surprise there is hubbub around closures and not around the XML-lang features. First, the closures proposal was floated as a PDF which is partway towards a spec in form, plus blogged about by Ahe, Bracha and Gafter, no less--people who know the platform and who know what they are talking about in language implementation (you can take issue with their choices in design). It also came out during a lull in the day-to-day Java world. The XML proposal was more or less buried among a slew of other JavaOne announcements, many of them ("open source!") broadcast quite loudly. The fact that Mark was one of the ones presenting it wasn't enough to make up for the slew of other happenings at the time. Most people I've asked about simply never heard about it.

  Besides, while some communities (like server app developers) could benefit from XML support closer to the language, there's a truth to the fact that XML is the agreed-on basis for all significant/interesting/new network data (meaning application data) protocols, including P2P, AJAX, the blog feeds, etc. It makes sense to make this easier to work with, and offering support through libraries alone is showing its limitations. I wanted to cry with joy the first time I saw E4X, it just made sense and seemed so natural to read and look at.

  Last, we have to let these people float their trial balloons. It's a well-known way of seeing what the response will be. There's zero commitment for any of these features. JSE 6 isn't even out the door yet.

  Cheers
  Patrick

  Posted by: pdoubleya on September 05, 2006 at 12:16 PM

• "Everybody has their favorite feature which they want to add"Wrong. Most people want the language to for a change stabilise and stay pretty much the same for a while instead of having a constant flood of new nonsensical stuff added to it purely for marketing reasons.
  Maybe that's what I'd like, a -cutthecrap compiler flag which disables all that crap.
  "That's the thing - web guys say nothing about XML"Of course not. We're not the ones wanting it, we're the ones happily using XML already and quite content with it. It's the people who 6 years ago heard that XML was all the hype and that in a few years "everything will be XML" and thus decided that "we MUST do everything in XML as soon as possible" who are all hyped up about it. And they're luckily few in numbers (but apparently in quite high positions in Sun's marketing (oops, product development) department.
  "There's zero commitment for any of these features" That's what we all thought when they first floated the idea of including SOAP servers, web servers, and database servers in the core API. How wrong we were, those too were decisions already made and nothing was going to change those unholy ideas.

  Posted by: jwenting on September 05, 2006 at 01:01 PM

• jwenting: The JCP is here for you to voice your opinion. When Java SE 6's JSR gets approved, this will be an approval from the community. (By the way, there is no database server in the core API... it's just part of Sun's very own version of the JDK.)

  Posted by: gfx on September 05, 2006 at 01:55 PM

• Romain,
  That's the thing that Geir already pointed out - Dolphin is being planned by Sun without having even filed a JSR for this. And when Mustang JSR is approved, it's not by the community (like in general election), it's by the representatives of the community. I don't have any prejudice against the current "plutocracy" process, but there's a difference between "approval from the community" and "approval from 19 members of the specific JSR group most of whom were approved by Sun to participate in this group in the first place".

  Posted by: kirillcool on September 05, 2006 at 02:00 PM

• I believe it's a bad idea to put every feature that any language on earth may provide into Java. It would be much better to streamline Java and to bring all the new Java5 features to the exisiting classes.
  While we're at it: modulize Java! It's already to broad and bloated. Nobody needs all the offered functionality in one project.

  So: No to closures and No to XML in the language!

  Posted by: luethjec on September 05, 2006 at 10:06 PM

• What we really need is a syntax for embedding blocks of any text without the need to quote each line, and full access to the AST tree by annotation processors. This way we could embedd anything, not just xml. E.g. I would be very, very, VERY happy if there was no need to quote each line of SQL.

  Posted by: joerg_wassmer on September 05, 2006 at 10:56 PM

• gfx, the JCP is pretty much a paper tiger in that, especially for the individual.
  Unless you're a member of a big group sponsored by a major company you have no real voice at all (you're just swamped out by more influential people). That's the consequence of the politicising of the process of course, and pretty much unavoidable. It's either that or a system where a single entity decides without any consultation whatsoever.
  I've watched the JCP closely for several years and that was the only conclusion I could come to, that unless you're part of a large corporate JCP team or a core member of the Sun team (which is probably the largest team anyway) you have little or no influence at all except maybe in some fringe JSR group that might at some point be considered once a corporate group takes an interest in it and injects their own people (replacing you as the driving force of that JSR).

  Posted by: jwenting on September 05, 2006 at 11:15 PM

• jwenting, I think it's much more complex than "the JCP is pretty much a paper tiger". A mainstream, commercially-oriented programming language has to get support from the corporate community to survive (and avoid forking), and Sun has been pretty good about that. The JCP is just one forum for Java's extension, and serves a particular purpose, which is to gain consensus and buy-in around what goes into the platform. But it's not the only way that Java is extended; most of the interesting work happens in the libraries, not the language, and the most interesting library work happens in open-source, not in the JCP. You're free to build, distribute and promote your own ideas and build a community around them, and those ideas may end up influencing the JCP, as dependency injection/IOC in Spring ended up influencing EJB 3. Besides, the fact there is a standard set of Date/Calendar classes doesn't affect the popularity of JodaTime, and there are popular alternatives to JSP/JSF/J2EE, logging libraries, regex, you name it.

  I think it's unrealistic to imagine that a language with as wide a range in deployment as Java could work in a democratic (one person, one vote) environment. Besides, some important individual contributors have emerged within the JCP, such as Doug Lea in the threading arena, and he did that based on his own skill.

  Normally I wouldn't defend the JCP so strongly, but I think one has to have a more nuanced view of the situation.

  Regards

  Patrick

  Posted by: pdoubleya on September 05, 2006 at 11:56 PM

• I really, really fundamentally dislike the idea of native XML; I just don't see that it adds enough to be worth the language shift.

  Closures, on the other hand, I like, although I agree that the impact of putting in into a mature and widespread language is troubling, particularly given the need for long-term backward compatibility.

  Frankly, I'd rather jettison that last part over the longer term and get on with langauge evolution - let's get rid of Enumeration and Vector in old APIs, we've held on to them long enough.

  Posted by: diathesis on September 06, 2006 at 08:51 AM

• You ask: "Does it add yet another way to do what can be done already with existing language constructs (this is what makes Swing so hard for the beginners)? Certainly."

  Existing language constructs cannot be used for control abstraction, but closures can. Your comment suggests to me that you really don't understand closures as proposed.

  Posted by: gafter on September 06, 2006 at 09:25 PM

• Neal,
  I don't want to sound too picky, but since Java is a Turing-complete language, everything that can be accomplished with control abstraction can be accomplished with current constructs. I think all the examples in your original proposal showed how closures make it simpler to achieve. I would greatly appreciate an example of a closure that shows a solution to an otherwise unsolvable problem.
  The examples that you show on your blog talk about simplifying the existing code by introducing a slimmed version of Strategy pattern. There are already ways to achieve this by using AOP, proxies and byte-code manipulations. They may not be as powerful as the proposed syntax, but they work.
  All of the examples of withLock and closeAtEnd indeed address areas that currently require some extra code, but are solveable with little extra effort in the current syntax. When i talked about introducing yet another way to solve common problems, these were the exact scenarios that i talked about. A Java programmer should know these problems and there's a correct solution that he should use. Introducing yet another (it's simpler, but it's no more correct) solution will introduce the same amount of multiple-solutionism (pardon my French) that Swing suffers from - too many correct ways to handle the same problem.

  Posted by: kirillcool on September 06, 2006 at 09:39 PM

• One doesn't measure expressiveness of a programming language by computability; as you observe, all of the languages we're talking about are turing-complete. You measure expressiveness by what a programmer can (or can't) abstract over. Closures allow you to abstract over blocks of code, which allows you to eliminate boilerplate in many situations. The alternative solutions you suggest replace one kind of boilerplate with another, and they break down for abstracting over many kinds of code blocks, for example when the block of code you're abstracting over contains break, continue, or return, or uses "this". Exception transparency is also difficult to achieve in those solutions.

  Worse, your suggested solutions require infrastructure outside of the Java programming language, which reinforces my point that the language doesn't support the kinds of abstraction possible using closures.

  Posted by: gafter on September 06, 2006 at 11:01 PM

• Neal,
  Which brings me back to the original point. Following your terminology, the asynchronous closures are no more than syntactic sugar to cut down the amount of code and the synchronous closures are glorified (and simplified) Strategy pattern to cut down on the boilerplate code. Which loosely means that you'd have another way to achieve the same thing that could already be achieved with a little boilerplate. I'm not saying that it's not a good thing, but one of the main reasons that beginner Swing programmers write bad code is that there are too many ways to complete the same task. Another glaring example would be the multitude of readers / streams / what nots to read a file...

  Posted by: kirillcool on September 06, 2006 at 11:21 PM

• You said:

  Mark Reinhold outlined the proposed changes to [JDK 7] at JavaOne 2005 [...] (no closures being mentioned at that time)

  Closures where in fact mentioned, see my version from Japan (has some info on super packages as well) or the original (if you log in, you can also listen to an recording of the talk).

  Posted by: peterahe on September 07, 2006 at 07:55 PM

• Peter,
  My bad. My intention was to say that the closures back then seemed much less mature proposal. Native XML support had an entire BoF devoted to it. I don't remember that closures were that "advanced" a year ago.

  Posted by: kirillcool on September 07, 2006 at 08:05 PM

• "I think it's unrealistic to imagine that a language with as wide a range in deployment as Java could work in a democratic (one person, one vote) environment." I fully agree Patrick, which is what I intended to convey. Doesn't remove anything from my assertion that the non-corporate contributor (or the contributor from small corporate members) has no say and that thus the idea that you can as an individual have any influence on the future of the platform by joining the JCP is a false one.
  Also doesn't remove the fact that the JCP is on the wrong track. Instead of being guardians of the language, protecting it against unwise change, they seem intent of introducing change for the sake of change. That's of course purely marketing driven, as being able to send out commercials speaking of a "new and improved" product are what sells new licenses (of appservers, IDEs, etc. etc.

  Posted by: jwenting on September 08, 2006 at 10:31 AM

• There is already a lot of callback strategies of any kind and they are all more ugly or weird or painfull than the others. Weither they are object functor schemes or method names in strings (yea it can be that bad) , we already have to live with the concept of an externally defined function and this is ok. The basics of that are intuitive for anybody with a valid coding permit. The only problem is that we have no standard and type-safe way to realize that concept.

  Posted by: lacroix1547 on January 07, 2007 at 02:33 PM

This work is licensed under a Creative Commons License.