Cay Horstmann's Blog

What's so Taxing about Return?

The Dual Role of return

Many programming languages get along just fine without a return statement. In Pascal, for example, the return value is assigned to a dummy variable whose name equals the name of the function.

function foo(arg: integer): integer;
begin
   (* compute return value *)
   foo := retval;
   (* maybe some more cleanup *)
end;

In Scheme, the body of a function is a sequence of expressions, and the value of the last expression is returned as the result of the function call:

(define foo (lambda (arg) expr1 expr2 .... retvalExpr))

And, of course, in assembly, the return value is simply deposited in a register :-)

movl retval, %eax

In Pascal and Scheme, you return to the caller when you reach the end of the function. There is no equivalent to the “quick and dirty”

if (somethingAbnormal) return null;
// more work in the normal case...

This shows that there are really two aspects of returning:

• what to return
• when to return

Closures

A closure is a block of code, packaged up for execution at a later point. When it executes, all the references to the surrounding code should just work as if the code had executed in the defining scope.

Here is a typical example, using the BGGA 0.5 syntax (which, like all proposal syntax, is highly subject to change):

public static void main(String[] args) {
    JFrame frame = new JFrame();
    JButton button = new JButton("Click me!");
    frame.add(button);
    int counter = 0;
    button.addActionListener({ ActionEvent e => 
        counter++; 
        frame.setTitle("Clicked " + counter + " times."); });
    frame.pack();
    frame.setVisible(true);
}

When the button is clicked, the closure gets called, the counter variable in the enclosing scope is updated, and the frame title is set to reflect the click count.

Wait, there is a problem here. By the time the button is clicked, main has terminated and the local variable counter is dead and gone.

Actually, though, the closure will capture a reference to a new int[1] containing the counter, and of course, a reference to the JFrame object.

All this can be done with any of the various closure proposals, by gussying up inner classes with the ability to capture non-final locals.

Unlike some other closures proposals, BGGA goes further and says that the closures also need to capture the meaning of execution transfer statements, i.e. break, continue, and return. (What about throw? That's never statically typed, so we don't expect to capture it.)

At first glance, this seems like an odd thing to do. When the action listener executes a break statement, surely we don't want to go back in time and revive the main method (at least not until the proposal to add continuations to Java :-))

But it comes in handy for another use of closures: programmer-provided control statements. Let's say I want to provide an easy way of iterating over a matrix:

for each(int i, int j: matrix) { // look, ma, no matrix[i].length!
    if (matrix[i][j] == 0)
        continue;
    . . .
}

This actually means: Pass matrix and the closure

{ int i, int j =>
    if (matrix[i][j] == 0)
        continue;
    . . .
}

to the each method:

public static void each(int[][] a, { int, int => void } block) {
    int i = 0;
    int j = 0;
    for (; i < a.length; j = (j == a[i].length ? 0 : j + 1), i = (i + ((j == 0) ? 1 : 0))) {
        block.invoke(i, j);
    }     
}

Of course, now continue should mean the right thing: continue after the block, with the next iteration of the for statement. (Sorry about the tortured logic in the for update; one must use an assignment, increment, method call, or new expression. I suppose I could use a closure invocation { => if (j < a[i].length) j++; else { i++; j = 0; }}.invoke()...)

The Point of No Return?

Back to the topic of returns. A closure returns a value (if it has a result type). For example,

{ int x, int y => Math.max(x, y) }

returns an integer, the max of its parameters. But if a closure contais a return statement, that means to return from the enclosing block. For example,

{ int x, int y => return Math.max(x, y); }

is a closure with return type void that, when invoked, causes its caller to return the max of the parameters (or, presumably, if the caller can't return an int, throw an exception).

Several commentators to my earlier blog point to this issue as the Achilles heel of the BGGA proposal.

More unhappily, the closure

{ int x, int y => Math.max(x, y); }

computes the max of its parameters, discards it, and returns no value.

When I heard about that, my gut reaction was fear...the fear of students queuing up for my office hours.

Ultimately, the culprit is the dual nature of return: yielding a value, and jumping to the end of the method code. In Pascal or Scheme, none of this is an issue. These languages have no return (or break or continue) to worry about.

Many Happy Returns

Let's try to throw some syntax at this. A BGGA closure body is a sequence of statements followed by an optional expression. Maybe that's too subtle. Let's make the return expression more prominent. Something like

{ int x, int y => int : stats => Math.max(x, y) }

I already see the line outside my office getting shorter. (I suppose it would also allow early returns from a closure, but I don't want to go there...That's what got us in trouble in the first place.)

But I have to agree with Stephen Colebourne that there are two entirely separate use cases here.

When one uses a closure for a control abstraction, the return type must be void since the closure denotes a statement. And it is pretty clear that return means to return from the enclosing scope.

When one uses a closure as a callback, to be invoked at a much later time, does one ever want to capture the enclosing semantics of return? I don't think so, but I will find out soon enough if I am wrong...

It would make sense to differentiate these use cases.

In a control abstraction, the programmer doesn't provide an explicit closure, but the compiler puts together a parameter list and a block. Conversely, when passing a closure to a callback, the programmer does the { ... => ... } thing. So, we can tell them apart.

In the first case, the block can contain return, break, continue, labeled break, etc. Pile it on!

In the second case, none of them should be allowed. It's just a syntax error. That should take care of the line outside my office. Students can wrestle with the compiler—what gets them is code that compiles and does the wrong thing.

This is almost the same as the RestrictedFunction interface, except that you can capture non-final locals. It is also somewhat related to the distinction in BGGA control abstractions. The for control abstractions allow break and continue, whereas other control abstractions don't.

If I understand the FCM/JCA proposal correctly, they have essentially the same solution. But the meaning of return changes from one use case to the other. I am not sure that's such a good idea. But again, it's just syntax.

I am a total amateur at this, of course, as I and the world are sure to find out from the blog comments in a few hours. But it seems to me that after the tweaks that are sure to come, BGGA will differ very little from FCM/JCS, except for the issue of method literals. (These may be nice to have for other purposes. I'll warm up to them if someone can show me how they solve my pet peeve: property boilerplate.)

Comments

• FCM/JCA distinguishes the two use cases based on whether or not the "control invocation" syntax is used to invoke the method. This raises a few issues. First, you can only pass a single method that has the synchronous semantics (never two or more). Second, you (Cay) expressed a personal preference last week that the control invocation syntax should be usable for the asynchronous use cases, which undermines that syntax being used to make the distinction. Third, the idea that you might want to "return" from a closure early surely is independent of whether or not the closure's result value is of type void or which invocation syntax is used, but in FCM/JCA those are indeed linked. Nevertheless, I support the idea that it ought to be syntactically easier to yield a closure's result, early or otherwise.

  Having the meaning of any component of a closure change meaning when you switch between the two invocation forms is a severe constraint on the types of refactorings you can easily do using closures, and will probably be a disaster for code readability (but a boon for authors of puzzle books). We're going to avoid that if possible. Google for "Tennent's Correspondence Principle" for a discussion.

  Posted by: gafter on April 16, 2007 at 02:40 PM

• 
  A good writeup, thanks. As mentioned, FCM/JCA distinguishes between callback-style and control-invocation-style.

  
  FCM is callback-style. A block of code can be assigned to a variable, or passed directly to a higher order method. We specifically name this feature an 'inner method' rather than a closure. It's semantics wrt return are those of an inner class method. This avoids the dreaded non-local return exception. We believe this works well and is familiar to the many existing Java developers out there, and is a natural semantic for Java.

  
  JCA is control-invocation-style, and an optional extension to FCM. Here the block of code appears to visually be part of a built in keyword, like foreach. With JCA, return will return from the enclosing method. There is no way to directly assign the block to a variable within the application code, and thus the opportunity for the non-local return exception is reduced. Again, the semantic for return is just logical in the context of Java, as it looks like a keyword, not an inner class.

  
  As you (Cay) note, this is two different semantics for return. And yet it works well in the context of Java where we have inner classes. Personally, I find the notion of 'RestrictedFunction' to be rather a nasty hack.

  
  Neal mentions that it is not possible to directly pass two closures to an API which both want to return from the enclosing method. My preferred answer is "Tough". There are possible solutions, but they make the overall result less safe.

  
  Neal also suggests that the use of return is separate from the calling syntax. This is a view based on implementing closures in Smalltalk or Ruby or Scala, where the language is designed around closures. I don't believe that Java is in that position - we need to respect what syntax and semantics work in a Java context.

  
  Oh, and method references (rather than method literals) won't help with boilerplate properties, but they will help with bean binding ;-)

  Posted by: scolebourne on April 16, 2007 at 04:05 PM

• Neal: I did indeed "express a personal preference" last week, out of complete ignorance. The question, if I recall correctly, was whether the audience preferred
  void launch(Executor ex) {
  ex.execute({ => doSomething(); });
  }
  or
  void launch(Executor ex) {
  ex.execute() {
  doSomething();
  };
  }
  At the time, I looked at the cosmetics and found the second piece of code comfortingly familiar. But I can change my mind in the face of new evidence :-) I am now thinking that the control invocation syntax should only be used when a method has been specifically tagged to accept it...unless, of course, I see new evidence.
  Maybe you have an example where the same closure (containing a return) should plausibly be usable in both an asynchronous invocation and a control invocation?
  Like I wrote, I don't think it is a good idea for the meaning of the closure body to change. I merely question whether a closure body containing return should be permitted in circumstances where it is highly unlikely to reflect the programmer's intent, such as in the action listener and executor examples.
  

  Posted by: cayhorstmann on April 16, 2007 at 04:34 PM

• Cay, do you have the freedom to choose another language to teach in order to make the line outside your office shorter?

  In my experience, you don't want the line to be shorter anyway. You want it to be a fairly constant length throughout, rather than suddenly grow before deadlines. That's why I set an assignment each week (or every two or three weeks), but then my students are on a partially-technical course anyway, so they need that hand-holding.

  I'd like us to consider multi-piece method names again in the context of BGGA and JCA. For example, given a Maybe<String>, where I've stolen Maybe from Haskell, you could do this:

  ifJust(String string: maybeString)
  {
  return string.toLowerCase();
  }
  else
  {
  throw new SomeException();
  }

  
  It wouldn't need to actually be the word 'else'. 'or', or 'otherwise' would be fine.

  Posted by: ricky_clarkson on April 16, 2007 at 04:54 PM

• "Maybe you have an example where the same closure (containing a return) should plausibly be usable in both an asynchronous invocation and a control invocation?"

  Cay, I'm not sure if this is what you mean, but:

  SwingUtilities.invokeAndWait()
  {
  textField.flibble();
  return textField.getText();
  }

  or a little pointless race:

  ConcurrencyStuff.fork()
  {
  int a=0;
  for (int a=0;a

  Posted by: ricky_clarkson on April 16, 2007 at 05:01 PM

• I'll try that code again.

  ConcurrencyStuff.fork()
  {
  for (int a=0;a<1000;a++);
  return "first one won";
  }

  for (int a=1;a<100000000;a*=2);
  return "second one won";
  

  Posted by: ricky_clarkson on April 16, 2007 at 05:03 PM

• Ricky: I'll bite on the Swing example. invokeAndWait is synchronous. In your case, there is little value in returning textField.getText(). That value can equally well be obtained in the enclosing scope.
  SwingUtilities.invokeAndWait({ => textField.flibble(); })
  return textField.getText();

  Moreover, is also not something one does a lot. Usually, you use invokeLater, where clearly you do not want to return a value into a method that quite likely has long exited. The library designers could even support this distinction, by tagging an invokeAndWait method as suitable for receiving a closure with return.

  Posted by: cayhorstmann on April 16, 2007 at 06:23 PM

• Cay, if you say that SwingUtilities.invokeAndWait() would return the same value inside and outsite the method just because it is synchronous, you do not fully understand EDT threading and why SwingUtilities' invokeAndWait and invokeLater exists...

  Posted by: mikaelgrev on April 17, 2007 at 12:44 AM

• Cay said: "Like I wrote, I don't think it is a good idea for the meaning of the closure body to change. I merely question whether a closure body containing return should be permitted in circumstances where it is highly unlikely to reflect the programmer's intent, such as in the action listener and executor examples."

  BGGA requires an error in these contexts, due to the fact that the received interface will extend the new marker interface RestrictedClosure. This is independent of which syntax is used in the invocation. My personal preference is that the control invocation syntax not be allowed when the received interface extends RestrictedClosure. That was the point of my question about which syntax you prefer for the asynchronous case; it seems most people disagree with me and think the control invocation syntax is fine.

  Posted by: gafter on April 17, 2007 at 02:19 AM

• Mikael: You are right. Since the Runnable executes on the event dispatch thread, it is utterly pointless to have a non-local return since it would result in a UnmatchedNonlocalTransfer exception. That further strengthens my argument that non-local return is not usually what you want outside of control invocations.

  Neal: Maybe most people disagree with you on the preferred syntax because they don' fully understand the implications? I certainly didn't when you asked the question. NB. I am a bit confused about the RestrictedClosure interface. Why are you restricting non-final locals?

  Posted by: cayhorstmann on April 17, 2007 at 06:44 AM

• Today in Java, there's no way to return from a method from within an expression. That's a big part of why I think the ability to do so with BGGA nested closures will be confusing. I side with FCM/JCA on this.

  As for multiple synchronous closures, I think this is an unlikely use case. I'd rather make the common cases clearer and easier.

  That said, I _still_ suggest expression methods as a way to solve simple use cases for this. See ECMAScript 4's plan, for example. In Java, expressions can't contain return, break, or continue, so that subject goes away.

  I also don't think that people are likely to be switching back and forth between different closure syntaxes. People are likely to code with a use case in mind and go that direction. If you switch syntax, it's probably also because you are switching what you are doing to start with.
  

  Posted by: tompalmer on April 17, 2007 at 09:46 AM

• By the way, invokeAndWait() can work great with non-local return. What you'd get is an InvocationTargetException which the helper method can easily unwrap into the exception that automatically becomes a return. No big deal.

  Posted by: tompalmer on April 17, 2007 at 09:49 AM

• Concerning non-final locals, I have yet to see how they differ meaningfully from fields in final locals (except the annoyance level). Non-final for either synchronous or asynchronous (and for concurrent or non-concurrent) is just fine given current status quo. The fewer annoyances on this point, the better, I say (unless you really ask for them badly with compiler switches or a separate analysis tool maybe).

  Posted by: tompalmer on April 17, 2007 at 12:54 PM

• For handling UI events, I prefer the superior technology offered by Actions. While my example is a little more verbose (I eschew the anonymous inner class - as a not particularly OO solution), I pick up a whole bunch of functionality 'for free'. Eg assigning an Icon to the button is trivial through the action, multiple controls (a button and a menu item for instance) in the UI sharing the same method (just assign them all the same action), enabling and disabling all of the controls which share the action at the same time (just enable/disable the action).

  For large UIs having an OO approach helps avoid the 'spaghetti code' problem. Also, since separation of concerns is a big deal these days, note that the creation of the button and placement on the UI is now separated from its functionality.

  Oh, and if we want to dynamically change the buttons functionality, it is as simple as setting a new Action, which replaces the previous one (without messing up other ActionListeners).

  By making them proper classes I can now organise my actions by using the packaging mechanism.

  It is all good. It is so good in fact (for such little effort), that anyone using anonymous inner action listeners really needs to step up and provide a justification for why they would do that when there is a clearly superior alternative.

  
  public static void main(String[] args) {
  JFrame frame = new JFrame();
  JButton button = new JButton(new FrameUpdatingAction("Click me!", frame);
  frame.add(button);
  frame.pack();
  frame.setVisible(true);
  }

  class FrameUpdatingAction extends AbstractAction {
  protected JFrame frame;
  protected int counter;
  FrameUpdatingAction(String name, Frame f) {
  super(name);
  frame = f;
  }
  public void actionPerformed(ActionEvent e) {
  counter ++;
  frame.setTitle("Clicked " + counter + " times.");
  }
  }
  

  Posted by: rickcarson on April 17, 2007 at 05:26 PM

• Rick: I totally agree with you that extending AbstractAction is much smarter than ActionListener. My code was just a toy example to make a point about closures.

  Posted by: cayhorstmann on April 17, 2007 at 08:16 PM

• @Cay,

  You make some very good points, not least because I agree with them :) In my post: Comparing Inner Class/Closure Proposals I too used the example of a return with and without a keyword and with and without a ; as a problem with BGGA.

  In my own proposal, C3S, I suggest that named non-local returns should be used. Your example (in BGGA):

  
  int someMethod() {
  ...
  x = someOtherMethod( { int x, int y => return Math.max(x, y); } );
  ...
  }

  
  That returns the Max from someMethod not someOtherMethod. In my C3S syntax you would do:

  
  int someMethod() {
  ...
  x = someOtherMethod( method { x, y => someMethod.return Math.max(x, y); } );
  ...
  }

  
  The trick is to name the method to be returned from if it isn't the inner most method. This seems to solve all the issues and is more flexible than the FCM proposal and isn't a special case (only inside certain structures), I propose that an inner class could use a non-local return.

  Once you have short syntax for any inner class you really don't need a separate control syntax. In addition to short inner class syntax I propose optionally: dropping return if it isn't ambiguous, dropping () if it isn't ambiguous, and dropping ; before }. Therefore the control example of a matrix might be:

  
  each matrix, method ( i, j ) {
  if (matrix[i][j] == 0) { return }
  . . .
  };

  
  

  Posted by: hlovatt on April 17, 2007 at 11:19 PM

• Cay,
  I am sorry but I still don't buy this closure frenzy.

  Personally even though I would preffer to use Action as quoted on previous post, to call an event handler I simply use the java.beans.EventHandler (existing since 1.4 !!!).

  This means the code would look like

  
  public class Foo extends JFrame{
  int counter = 0;
  public Foo(){
  JButton button = new JButton("Click me!");
  this.add(button);
  button.addActionListener(EventHandler.create(ActionListener.class, this, "doStuff"));
  }

  public static void main(String[] args) {
  Foo foo = new Foo();
  foo.pack();
  foo.setVisible(true);
  }

  public doStuff(){
  // get the data from the context and do what it is suppose to do
  }

  }

  Isn't this much more simple ? Plus built with existing features !

  I ask to all the closure lobier to provide real world usefull examples that shows noticeable improvement effects and a major advantage of introducing a "closure" (improper name IMHO) feature into the Java language.

  To me, with all the functionality existing : anonymous class, inner class, even handler, proxies, genericity ... already existing in Java, at this time, there is no need for closure (until I'm shown a realy usefull example).

  If we need something it is : either a runtime aware genericity (something you can do dynamicity onto and get the realy parametric type at runtime to do automatic/generic tasks based on this : display, edit, ....) or AOP (EJB3 introduced early AOP to Java, we need a platform solution that will unify this from client side to server side).
  

  Posted by: bjb on April 17, 2007 at 11:33 PM

• Oops typo, sorry.

  Should have said:

  x = someOtherMethod( method( x, y ) { someMethod.return Math.max( x, y ); } );

  Which if you choose to drop superflous () and ; then you could write:

  x = someOtherMethod method( x, y ) { someMethod.return Math.max x, y };

  Posted by: hlovatt on April 18, 2007 at 12:03 AM

• @bjb & others

  You are right - the new inner class proposals including my own C3S add little other than syntatic sugar. Compared to some of the other prosals I think one of the strengths of C3S is a simple mechanical translation to existing code. So your example with minimal changes would be:

  public class Foo extends JFrame {
  private int counter = 0;

  public Foo() {
  final JButton button = new JButton( "Click me!" );
  add( button );
  button.addActionListener( method( notUsed ) { doStuff() } );
  }

  public static void main( final String[] notUsed ) {
  final Foo foo = new Foo();
  foo.pack();
  foo.setVisible( true );
  }

  public void doStuff() {
  // get the data from the context and do what it is suppose to do
  }
  }

  With more changes( dropping () and ; and using method more):

  public class Foo extends JFrame {
  private int counter = 0;

  public new {
  final button = JButton.new "Click me!";
  add button;
  button.addActionListener method( notUsed ) { doStuff }
  }

  public static method void main( final String[] notUsed ) {
  final foo = Foo.new;
  foo.pack;
  foo.setVisible true
  }

  public method void doStuff {
  // get the data from the context and do what it is suppose to do
  }
  }
  

  Posted by: hlovatt on April 18, 2007 at 01:50 AM

• The truth is that closures are much more atractive as custom iteration and as arguments of custom callbacks that do something extra than as a replacement of the swing action structure, since this is already at the point that is more usefull than a callback could ever could be. I'd like that the actions listeners where made private api, but that is another story.

  Posted by: paulofaria on April 18, 2007 at 03:26 AM

• Concerning C3S, I love concise language, and I love Ruby. But changing the Java language dramatically is a bad idea in my opinion. C3S would be better planned out as a different language, and I doubt I'm alone in this opinion.

  Concerning Swing events, I agree that Actions are probably better than spaghetti closures. But I also agree with others that there are plenty more use cases for closures than that. (Like almost nonstop use cases in everyday code.) I also strongly dislike the use of reflection such as used by EventHandler. Static usage analysis is nice, and this makes a mess of that.

  As an experiment, here's an attempt at using a mock CSS/XBL-ish Swing utility library for Java 7 closures and properties:

  
  styler.addRule(Style style: ".stuff") {
  style.onAction(#{doStuff()});
  style.enabled = false;
  style.fontWeight = BOLD;
  style.icon = myCoolIcon;
  }

  
  That probably needs more careful thinking to make it good, but it's not very hard to understand, and it's not so spaghetti either.
  

  Posted by: tompalmer on April 18, 2007 at 09:19 AM

This work is licensed under a Creative Commons License.