Skip to main content

And so it begins - the first closures puzzler

Posted by kirillcool on February 5, 2008 at 10:20 AM PST

Neal Gafter has posted the first closures puzzler. I guess the second edition of Java Puzzlers is in works, and closures will be a hefty addition to the book. If anything, this makes me really sad.

I was very excited to lay my hands on the first edition, but after reading through a few chapters, i skimmed the table of contents, glanced at the visual illusions and never came back to it. If i had to sum this book in one sentence, it would be "great optical illusions and irrelevant content". I know, this is a harsh statement, and i have nothing but respect to both authors (in fact, i consider "Effective Java" to be the book that i would take with me to a deserted island on a condition that it has WiFi access).

I've been using Java for the last 8 years, most of it exclusively (amounting to about 10-12 hours a day, including the work and the side projects). I've written my share of new code, and i most certainly have seen my share of old code that i had to maintain, fix and extend. But never once have i encountered anything even remotely connected to any Java puzzler presented in the book and at the conferences.

Bit-level shifts, integer overflows, overloaded methods, reflection, generics - you name it. I've looked at the examples, i've tried to match them to the real code that i see during the day, and i have found nothing. Of course, i am looking at the tiniest sliver of Java code that exists in the enterprise sector, but somehow i get an impression that i'm not alone. I would even go as far as to say that a feature implemented in a way that results in at least one puzzler is not a feature worth having in the language.

Which brings me to the quote of the day. It comes from an announcement on the end-of-life for Ruby.NET:

As a researcher, my prime interest is not in developing products, but in developing innovative new ideas and having an impact by having those ideas used in the real world.

The generics were added to the language in a very incomplete manner, mainly due to the restrictions on binary compatibility. If we let the researchers in us (even if they are the best researchers) to do the same with closures, i'll pass.

Related Topics >>

Comments

@hlovatt, now add a "CHERRY" to the Flavor enum...

@ 5er_levart, In the difficult recursive cases the two enums will always be instep with an exact one to one correspondence. For example, the example doesn't work if you don't add CHERRY_RED s well as CHERRY - otherwise what does RED return. When there is not an exact one to one between two classes you are unlikely to have a problem. The whole example is fairly contrived, though not impossible.

@Remi, Thanks for the advice - I was not aware of that but it is correct that it does so - I didn't think that one through. Best make a local, static, copy then: enum Flavor { CHOCOLATE, STRAWBERRY, VANILLA; private static final Color[] colors = Color.values(); Color color() { return colors[ ordinal() ]; } }

@hlovatt , beware of values(), it makes a defensive copy. So the array is cloned each time flavor() or color() is called.

Rémi

Personally I would be tempted to go with:

enum Color { BROWN, RED, WHITE; Flavor flavor() { return Flavor.values()[ ordinal() ]; } } enum Flavor { CHOCOLATE, STRAWBERRY, VANILLA; Color color() { return Color.values()[ ordinal() ]; } }

Rather than use closures, which seems an overly complicated solution to my eyes.

I really appreciate the puzzle. The first thing I thought when I saw Neal's post was that it was odd he was attempting to demonstrate how closures could introduce new puzzlers - and my second thought was 'I bet he's up to something'. Of course, the joke is that closures behave exactly as expected - Neal is making a case for closures, and getting people to download and play with his prototype implementation to boot!

I especially liked that 5er_levart came up with a solution to the problem using closures themselves... good stuff!

Roberto

@jwenting: According to a post on the Concurrency interest mailing list by Doug Lea, it was the parallel api use case that came first and prompted the closure proposal.

@jwenting - I didn't know C was a language "designed around closures/function pointers" :). Also, I think closures and function pointers are two entirely different things and we don't mix/compare them. I don't see the closures proposal as "retrofiting" anything to Java language. Closures are a natural fit for Java. Java is just missing them. Java will not loose anything with closures. People will still be writing procedural/OO code where they see it fit and more natural (the majority of code). And people will resort to closures/function types where they find anonymous inner classes that they write today awkward to use and difficult to read. Tons of code will still be written in Java and will have to be maintained, but that code will be more readable with closures in places where anonymous inner classes would make it obfuscated. I see you are skeptical about it. I challenge you to "fix" the problem with cross-referencing enums without using closures and post the solution here. Let's compare it with the "fix" I posted above and see the "obfuscation" level of the fixes. I'm not suggesting that this is an example of real-world code. But the problem of "circularity" of dependencies does pop-up quite often in real-world and is not being tied to static class initialization only. Closures can help here making code more readable. I sense some king of "fear of closures" in the posts. They are actually quite harmless and compared to generics quite easy to grasp. Just try them (the prototype) and you'll see it doesn't take you a minute to start using them without reading the spec.

" Swing wasn't retrofitted with generics in JDK 5.0, what makes you think that they'll retrofit it with closures in JDK 7.0? " Or to use a more vexing example, Swing wasn't retrofitted with ArrayLists when those were introduced in Java 1.1, it uses Vector to this day despite Vector being effectively deprecated.

Closures/function pointers are nice in languages designed around them, languages like C and Ruby. Retrofitting them into Java (or any language that's not designed around them) however is a very bad idea. You're not going to be retrofitting your Phillips screwdriver to work as a torque wrench, are you?

And worse, the entire idea was launched to please people who screamed that Java is dead unless closures are added, purely for marketing reasons. The "use cases" were made up after the fact and are so obscure that it's quite clear that hardly anyone will have real use for them, but they'll be massively abused, creating tons of work for people having to maintain the utterly unreadable and obfuscated code that will result.

@krill: The generics were added to the language in a very incomplete manner, mainly due to the restrictions on binary compatibility. If we let the researchers in us (even if they are the best researchers) to do the same with closures, i'll pass.

Amen to that! If we should be doing anything in Java7 it is to finally fix Generics.

@mikaelgrev He probably does - but it doesn't mean that his proposal is correct. All I can do is look at what closures are in other languages (and the subsequent programming possibilities) and look at CICE and say it doesn't match what everyone else calls a closure. http://ducktyper.com/2007/12/28/closures

@ jkuhnert You are sopt on, my argument is not technical, it is from a human/statistics perspective. Technically I am out of my league compared to Neal...

@mikaelgrev: I have no idea how to respond to your last statements as they are more about people and less about the technical merits of the proposal....I'm much worse at being "human" and would probably tend to say "f-em if they don't like it" because I'm an arrogant jerk...it seems like lots of java developers are getting in to dynamic languages just fine to me though.

Going with the "opinion based on evidence" route the current industry buzz surrounding NBL / Groovy / JRuby / Ruby / Scala all seem to indicate that java developers are capable of not being mediocre after all. Which - in theory - supports the idea that at least some segment might welcome it.

@Ricky > If it was average programmers designing the language, we'd be in a lot worse trouble!

100% true. But the target programmers should be average Joe (mathematically more correct called "Median Joe"). Empathy (the ability to see what someone else is feeling/thinking) is not proportional to intelligence. What I mean is that it is not implied. That is why it is important to let great, smart and highly educated people like James, Neal and Josh construct proposals and let us lesser beings be the executive judges.

(Rant mode on). Apple has understood as of late how things should be run. Let designers who are in touch with the audience decide what the engineers should construct. In language design I get the feeling that it is the engineers that run the show because the guys in touch with the audience does not have a PhD.

@ jkuhnert No, Java developers are pretty smart, even smarter than the president I'd say! ;)

Seriously, no, developers are in average not less or more smart depending on language. But, when you learn a new language you study it deeply and adjust your way of doing things to the tools provided by that language. Then, people are different. Some like few characters, and may choose perl, some like it wordy and go for xxx. Control types go for pure assembler. Java has a big developer base and many of these developers have gotten used to do things in a certain way. It always takes a bigger efforts to re-learn something than to learn something new from the beginning. This is why I think that we should just enhance the Java way and not add a completely new paradigm of doing things. The most obvious argument against that last sentence is "those who are not willing to re-learn should not be developers". True, maybe, but we live in a reality where developers will be upset that they must go back to school, so to say. Think of generics, it is a much lesser leap than BGGA closures, yet people bitch and moan about it.

People that want to abandon Java because the lack of closures should do so. Honestly, it is better for all, and I say that without disrespect. Scala is a good alternative but Groovy and Ruby are as well. I do not understand why leaving Java is such a big deal. Those who want closures, continuations or lesser type stringency should choose a language that fits their needs better. There is no need to take Java to where they want to be. Java is all about the API anyway. Scala/Groovy with the JDK API is still 90% Java to me.

I see nothing negative with using several languages. Java is not holy. Use whatever fits your needs.

In conclusion, the geek within me say "go for BGGA" but the strategist within say "go for least friction unless you have good will to pay with". For things like this I tend to listen more on the strategist within (and satisfy the geek by doing yet another late night project on something cool ;)

mikaelgrev: There is usually a gap between language designers and average programmers.. if that were not the case languages would be a lot more cludgy, because it takes a good programmer to design a general-purpose language well.

If it was average programmers designing the language, we'd be in a lot worse trouble!

@mikaelgrev So - are you saying that generally most ruby or programmers are generally smarter than your average java developer and that's why it's bad? Ie Java is intended for mediocre programmers only? Many of us lowly java programmers are getting very close to abandoning the language entirely because of a lack of support for eliminating repetitive code all over the place - something that this closures proposal would certainly help. I would hope that we'd try and let the community "live up to our expectations" vs. just assuming that they are all too dim witted to get it.

> Do you think it's possible these guys are a little more experienced with language design than your average experienced java developer?

I do, but I think that is part of the problem and not a defence. They are intellectually and educationally too far from the median of the 5.000.000 Java developers out there.

@ jkuhnert - I have a feeling that Joshua Bloch knows how to define a closure better than both of us... Read the CICE proposal, note the headline: "Concise Instance Creation Expressions: Closures without Complexity"

http://docs.google.com/View?docid=k73_1ggr36h

Just think that with closures a gazillion listener interfaces and classes will be gone! If they retrofit Swing the JDK for the complete removal of those Listener things then it would be paradise! But they won't, and we will end up with a big mess of legacy code. This would go for the collections API too. People dreaming of dropping iterators and for-loops when dealing with collections and using closures instead will be disappointed. All the collections are interfaces, and therefore methods to support closures can't be added without breaking backward compatibility.

C'mon guys. I liked the puzzler. I happen to enjoy testing my knowledge with puzzlers like that one. It's not that I would like to create cross referencing enums in my programs on a daily basis, but it's useful to know that the program in this puzzler can be modified with not a single drop of sweat so that it does not suffer from the unexpected behaviour. An guess what? This can be accomplished elegantly using closures... enum Color { BROWN({=>Flavor.CHOCOLATE}), RED({=>Flavor.STRAWBERRY}), WHITE({=>Flavor.VANILLA}); private final {=>Flavor} flavor; public Flavor flavor() { return flavor.invoke(); } Color({=>Flavor} flavor) { this.flavor = flavor; } } enum Flavor { CHOCOLATE({=>Color.BROWN}), STRAWBERRY({=>Color.RED}), VANILLA({=>Color.WHITE}); private final {=>Color} color; public Color color() { return color.invoke(); } Flavor({=>Color} color) { this.color = color; } }

@jwenting: Considering Neal's experience and James having helped to create Java what makes you think your opinion is really "definitely" correct on the closures proposal? Do you think it's possible these guys are a little more experienced with language design than your average experienced java developer?

@mikaelgrev: those aren't closures - they're anonymous inner classes...

Btw, I am for closures, the CICE kind (with ARM blocks).

yeah, I'm with thiagosc. This debate is getting boring now. Can't wait to see it added to the jdk.

So, where're the closures?
As far as I can see there're only function pointers there. Is that all? Can anybody post a real example of a Java closure, please?
Thanks, Antonio

Just think that with closures a gazillion listener interfaces and classes will be gone! If they retrofit Swing the JDK for the complete removal of those Listener things then it would be paradise! But they won't, and we will end up with a big mess of legacy code.

This is exactly the point that i address in my next entry. Nothing will be gone, most certainly not at the language level. Do you really think that once closures are in JDK 7.0, the support for inner anonymous classes will be removed from the language? Swing wasn't retrofitted with generics in JDK 5.0, what makes you think that they'll retrofit it with closures in JDK 7.0?

Sometimes I think Sun listens too much to their target audience. They don't know what they need and it's your job to find it out before the competition does and provide it. Just put the damned closures there and don't ask questions.

People will bitch and moan even if you do nothing, just by the simple fact that it has become fashionable to bash Java even for the silliest reasons. If no addition is made then Java is dead, if things are improved then Java is too complex. On the other hand Microsoft transforms C# into a Frankenstein and the same bloggers applaud it.

Just think that with closures a gazillion listener interfaces and classes will be gone! If they retrofit Swing the JDK for the complete removal of those Listener things then it would be paradise! But they won't, and we will end up with a big mess of legacy code.

so even the biggest proponent of closures (read, function pointers) has to admit that they're bad, unintuitive, obscure, and lead to obfuscated and error prone code. (That's Neal obviously, not Kiril). Now if Neal were only enough of a sport to publicly retract his proposal and his advocacy for the entire monstrosity maybe some sanity could be restored to the JCP and the direction of the development of the Java language.

Yes! Indeed how could Kirill write anything against bad things without even doing them? :-) What about the professional intuition (which normally comes with professional experience)?

Kirill: I can't understand why you would blog against this puzzler (and specifically against closures) without having tried it first. You might as well blog saying "XYZ doesn't interest me" and leave it there. I'd like to understand why, because I used to like reading your stuff, specifically about Swing.

Hi Kirill, I wonder wether you have ever thought about closures during your last 8 years... ;-)

For outsiders, mathematics has one whopper of a puzzler: we can prove that we can't prove the consistency of all but the most trivial of mathematics. Similarly in programming, systems that do not admit apparent paradoxes are likely to be of limited use.

Re: "And by the way, if this is what the current implementation of enums does (for whatever reasons, be it backwards compatibility, technical limitations, scoping during the release cycle), then it is contrary to what the code implies and i still stand by my words about features that make room for puzzlers."

The puzzle has only a little to do with enums as well! The puzzle is mainly about order of static initialization, and it is an issue that was present in the earliest versions of the language. While I agree with you that the Java programming language might be better without static initialization, it would hardly be anything like the Java we know today.

Oh, c'mon... If you couldn't have features that cause puzzlers in a programming language, no language would exist at all! Or, at least, no one worth using, because it would be so simple you couldn't do anything interesting with it.

And by the way, if this is what the current implementation of enums does (for whatever reasons, be it backwards compatibility, technical limitations, scoping during the release cycle), then it is contrary to what the code implies and i still stand by my words about features that make room for puzzlers.

And so eventually it boils down to this simple example that has nothing to do with closures: public class Puzzler { public static void main(String[] args) { int colorsWithFlavors = 0; for (Colour c : Colour.values()) if (c.flavor != null) colorsWithFlavors++; System.out.println("Colours with flavours " + colorsWithFlavors); int flavorsWithColors = 0; for (Flavor f : Flavor.values()) if (f.color != null) flavorsWithColors++; System.out.println("Flavours with colours " + flavorsWithColors); } } enum Colour { BROWN(Flavor.CHOCOLATE), RED(Flavor.STRAWBERRY), WHITE(Flavor.VANILLA); final Flavor flavor; Colour(Flavor flavor) { this.flavor = flavor; } } enum Flavor { CHOCOLATE(Colour.BROWN), STRAWBERRY(Colour.RED), VANILLA(Colour.WHITE); final Colour color; Flavor(Colour color) { this.color = color; } } Undoubtedly, an obscure JLS sub-sub-sub-paragraph will shed light what happens with enums that cross-reference each other when you try to call the values() method. After using enums sparingly for the last two years, i must admit that i have never found myself in a situation when i needed cross-referring enums. Maybe it's just me. Maybe everyone else programs their entire codebase in enums...

Part of the art of writing a puzzle is misdirection, and it appears that you've fallen for it hook, line, and sinker - i was once excited at the "one-upmanship" games during the puzzler presentations. Like i was excited following all the intricacies of multiple inheritance, operator overloading and pointers...

Re: "Is this puzzler on enums or on interaction between enums and the proposed closures?" Find out for yourself. If you can't be bothered to read and understand the issue, how can you draw any competent conclusions from it? Part of the art of writing a puzzle is misdirection, and it appears that you've fallen for it hook, line, and sinker.

Neal, without going into the details of your closure proposal, as you are clearly in a much better position to analyse its interaction with existing Java constructs, i have one question. Is this puzzler on enums or on interaction between enums and the proposed closures? If it's the later, then i stand by my words - the feature as proposed introduces a puzzler where it hasn't been before. If it's the former, why do you need to involve closures to show a puzzler on enums?

Thanks, Kirill

Re: "a feature implemented in a way that results in at least one puzzler is not a feature worth having in the language" In that case, when you understand the puzzle and its solution, you're likely to conclude that enums are not worth having in the language. As for the puzzles in the Java Puzzlers book, you're going to want to toss out inheritance, constructors, exceptions, arrays, threads, integers, ... well, just about everything.