Scott Violet's Blog

Should we generify Swing's filtering and sorting?

I was tempted to name this blog Tricks and Subtleties of generics but I figured more people would be interested in providing feedback on mustang sorting. Anyway...

One of the bigger features we've added to Swing for the 1.6 release is sorting and filtering for JTable. The sorting and filtering API were designed in a way so that it is not tied specifically to JTable. In other words in a future release the same APIs will be useful for JList, JTree and possibly JComboBox. If you haven't played with it yet head over to the JDK development page for mustang and download the latest bits. Or if you just want to look at the documentation for the major classes, look to RowSorter, DefaultRowSorter, TableRowSorter and RowFilter.

We took a distinctly different path with filtering/sorting than with similar aspects of Swing. In particular RowFilter takes an instance of Entry that is meant to wrap the underlying model. This allows for instances of RowFilter to be used regardless of the underying model. This is useful in that you could create a filter for regular expressions that would be useful regardless of the underlying model type. This is different from the renderers in which you have a distinct interface for each component type.

Here's a quick example that will only show rows where some column contain 'foo':

  TableRowSorter sorter = new TableRowSorter(tableModel);
  sorter.setRowFilter(RowFilter.regexFilter("foo")); *
  table.setRowSorter(sorter);

* In current mustang builds (as the time of this blog) the regexFilter uses match for inclusion, it's going to be changed to use find. Before this change is done this example would need to be ".*foo.*".

While RowFilter doesn't directly take the underlying model, you can dig if out if you need to. Here's the example in the javadoc:

  RowFilter ageFilter = new RowFilter() {
    public boolean include(Entry entry) {
      PersonModel personModel = (PersonModel)entry.getModel();
      Person person = personModel.getPerson((Integer)entry.getIdentifier());
      if (person.getAge() > 20) {
        // Returning true indicates this row should be shown.
        return true;
      }
      // Age is <= 20, don't show it.
      return false;
    }
  };

In the mustang forums Remi has suggested we consider generifying RowFilter. The hope with generifying is that it would avoid the casts, as well as adding type safety.

Here comes the fun, and were I would like your feedback.

RowFilter takes an instance of Entry (static inner class of RowFilter). Generification would have to happen here. One possible way to change it would be to go from:

  public static abstract class Entry {}

  public static abstract class Entry<M, I> {
    public abstract M getModel();
    public abstract I getIdentifier();
  }

But to be useful RowFilter must also be similarly generified:

  public abstract class RowFilter<M,I> {
    public abstract boolean include(Entry<M, I> entry);
  }

And DefaultRowSorter would be generified around the type of model:

  public class TableRowSorter<M extends TableModel> extends DefaultRowSorter<M> {
    public M getTableModel();
    public void setRowFilter(RowFilter<M,Integer> filter);
  }

After all this the example in the javadoc would become:

  RowFilter ageFilter = new RowFilter<PersonModel,Integer>() {
    public boolean include(Entry<PersonModel,Integer> entry) {
      // Notice the cast to PersonModel is gone
      PersonModel personModel = entry.getModel();
      // And the cast here is gone to
      Person person = personModel.getPerson(entry.getIdentifier());
      if (person.getAge() > 20) {
        // Returning true indicates this row should be shown.
        return true;
      }
      // Age is <= 20, don't show it.
      return false;
    }
  };

Notice the casts are gone (although one could argue we've just shifted them to the constructors), and you have type safety. You couldn't set this filter on a rowsorter that was using a different model or providing a different identifier. Sounds good, right?

Well, turns out this generification isn't quite right. In particular a RowFilter that doesn't require a specific model or identifier would no longer work with this generification. For example, you couldn't use a RowFilter<Object,Object> with TableRowSorter<PersonModel>, which isn't right. Back to Gilad's tutorial for another pass.

A RowFilter should be able to filter any models that extend the declared model, or likewise extend the declared identifier. This implies:

  public abstract boolean include(Entry<? extends M, ? extends I> entry);

And similarly the setter for the RowFilter should allow any filters that can handle a superclass of the specified model, or superclass of Integer. For example, you should certainly be able to use a RowFilter<Object,Object> with a TableRowSorter<PersonModel>. This implies:

  public void setRowFilter(RowFilter<? super M,? super Integer> filter);

That leaves us with RowFilter as:

  public abstract class RowFilter<M,I> {
    public abstract boolean include(Entry<? extends M, ? extends I> entry);
  }

TableRowSorter as:

  public class TableRowSorter<M extends TableModel> extends DefaultRowSorter<M> {
    public M getTableModel();
    public void setRowFilter(RowFilter<? super M,? super Integer> filter);
  }

So, this will certainly allow you to have more type safety in your RowFilter, at the cost of readability. What do you think?

Other generics quirks:

As part of this change DefaultRowSorter will also be generified around the model type. DefaultRowSorter makes use a private inner class to aid in sorting:

    private class Row implements Comparable<Row> {

DefaultRowSorter creates an array of these:

  Row[] viewToModel = new Row[size];

But once I gerified DefaultRowSorter this array creation didn't work. In particular it produces a compile error. Why you ask? Turns out you can not create an array of a generic class and as DefaultRowSorter is considered a generic class, so is Row. The fix is to make it static and take an instance of the outer class:

  private static class Row implements Comparable<Row> {
    public Row(DefaultRowSorter sorter, int index) {}
  }

Painful and obscure to say the least. Alternatively if I were using a collection class, say an ArrayList<Row> this would not have been an issue.

Special thank to Peter von der Ahe and Scott Seligman for helping out with my various generics questions and proof reading. And another thanks to Graham for quickly pointing out I missed escaping all the < and >s.

    -Scott

Comments

• I find it hugely confusing when you write generics like this:

  class TableRowSorter<M extends TableModel>

  Please don't do that. 'M' is not _completely_ generic and deserves a better description. F.E.

  class TableRowSorter<ExtendedTableModel extends TableModel>

  See how method signatures also become more clear:

  public ExtendedTableModel getTableModel();

  Java never used to use single character variable names for _really good_ reasons. The current trend to use (virtually) meaningless single character names is _very_ hard to read.

  Honestly, did anyone write code like this before generics came around? What is clearer:

  public ExtendedTableModel getTableModel()

  or

  public M getTableModel()

  I'm not blaming you for following convention - I simply believe the convention itself is one of the hugely negative aspects of generics. It would be a shame if your nice generic filtering and sorting capability was incomprehensibly documented.

  Please do not use single characters for generic types. I can't parse it.

  Cheers.

  Posted by: markswanson on August 25, 2005 at 05:38 PM

• Mark,

  Type variables are different from type literals as String variables are different from String literals.

  Therefore to use the same naming conventions for both type variables and typel literals is confusing, that is why the Single Uppercase naming convention came into being. Whether that was the best choice is arguable, but whatever the convention is, it needs to be distinct from type literals.

  So when you say "Please do not use single characters for generic types. I can't parse it" the problem is your's sorry.

  Scott is following conventions, and if you can't parse that, you'd do well to put some effort into learning how to.

  Once you have grokked it, then by all means try to find something better, but I fear you will have an uphill battle trying to get anything else accepted as standard.

  Of course, you could argue that Scott should have shown the javadocs which document the type variables and give some explanation to the type Variables.

  Bruce

  Posted by: brucechapman on August 25, 2005 at 07:21 PM

• Frankly, I don't care if the implementation of the library gets somewhat bulky if the user code gets cleaner.

  So: I fail to image how the last change would make the user code clumsier than the first attempt. But I see how much cleaner it gets be using the correct types in first example.

  I'd go for generify the filter-stuff...

  Posted by: javajk on August 25, 2005 at 11:51 PM

• Scott,

  Sun has made the choice to add generics to the language I think that it should be adopted wherever it makes sense. Generics are complicated in some parts, yes, but it will stay complicated if we (the developers) are not using it regularly.

  So, generify it. It will hurt for some but the pain in no less than if you just use generics _sometimes_. Hope you understand what I mean.

  Cheers,
  Mikael Grev

  Posted by: mgrev on August 26, 2005 at 05:03 AM

• markswanson and all,
  Sorry for not including all doc. This is a work in progress and I wanted to get the feedback before cleaning and sanitizing it all. Rest assured that if we go this route the doc will explain this in all it's gory detail.
  Thanks for the feedback so far folks,
      -Scott

  Posted by: zixle on August 26, 2005 at 07:43 AM

• Yes, I think generics should be used whenever they are possible. If someone doesn't want to use the generic classes they can use them raw (without type parameters) and ignore them completely.

  Posted by: keithkml on August 26, 2005 at 08:07 AM

• Just to clarify, I'm a big supporter of generics and would like to see the filter/sort solution gerericized (if that's a word). I've converted large amounts of code to use generics and use generics in all of the new code I write. I'm not new to it and have considerable time and experience at trying to parse generics.

  I realize it's an uphill battle because of the style used in Sun's javadocs - but I'm sure of one thing: single character conventions are a mistake. With a descriptive name I can tell at a glance what something is - with a single character I have no idea and must take a cumbersome look elsewhere for the definition.

  Perhaps a discussion on another thead about a better descriptive convention would be more constructive than my complaining:_)

  Cannon fodder: prepend "GT" for GenericType.

  Cheers.

  Posted by: markswanson on August 26, 2005 at 08:16 AM

• OK, I should have given an example:

  class TableRowSorter

  public ExtendedTableModelGT getTableModel();

  instead of

  public M getTableModel();

  If there is a forum that would be more appropriate to discuss this please let me know.

  Cheers.
  

  Posted by: markswanson on August 26, 2005 at 08:27 AM

• markswanson,
  I would suggest creating a new topic on the mustang forums: http://forums.java.net/jive/forum.jspa?forumID=25 . I know various language folks read that.
      -Scott

  Posted by: zixle on August 26, 2005 at 09:03 AM

• I'd say Mikael is right on this one.

  Some of us might think the whole situation unfortunate. I'm still mixed on the generics thing, myself. But I think Mikael has hit the nail on the head.

  Posted by: tjpalmer on August 26, 2005 at 09:45 AM

• Yes, do generify the API.
  Regarding convention for generic type parameters, I don't think single capital letter is bad. Why?
  a. there are usualy just a few of them per class (1, 2, 3 or 4 at most)
  b. If they are constrained:
  M extends TableModel
  then the declaration gives you all the info you need.
  c. when using the generic type you usualy instantiate a particular concrete type:
  new TableRowSorter<MyTableModel>()
  then modern IDEs (like IDEA) will give you a hint when typing in the method name and tell you something like this:
  public MyTableModel getTableModel()

  Posted by: 5er_levart on August 26, 2005 at 10:53 AM

• MarkSwanson:

  You really have no idea how quickly long parameter names can get way out of hand. In many declarations involving multiple generic parameters, what readability you gain by using long names you quickly lose by having to split declarations across multiple lines. Well named generic classes with long type names can easily start running into 40-50 characters long, which is way too long to be useful. Even the javadoc (especially the javadoc, actually) starts to get very difficult to parse in some places when you try to be descriptive with generic parms in the same way that we all generally try to be with variables.

  When you work with generified API's, typically the short typenames have a common meaning across many many classes. You'd be suprised how quickly you learn these meanings as you work with the API in question. In this case, the cost of making the API 'easier' for the uninitiatied is fairly high for those that are working directly in the code everyday.
  

  Posted by: david_hall on August 26, 2005 at 07:35 PM

• Scott:

  A Big +1 for generifying filters and sorters (and models, and ...)
  

  Posted by: david_hall on August 26, 2005 at 07:47 PM

• I think the generification is a good idea--Patrick

  Posted by: pdoubleya on August 27, 2005 at 12:04 AM

• +1 for generification

  Posted by: doctorture on August 27, 2005 at 05:28 AM

• I strongly support generification of these classes. Even though it makes their source code harder to read, it makes client code easier to read and more typesafe (as you mentioned). Has anyone waded through the latest versions of some of the Collections classes? Nasty exercise, but it makes them much easier and safer to use. If possible, all "required casts" should be eliminated from Swing client code by adding generic declarations to Swing's API.

  Posted by: tball on August 27, 2005 at 09:28 AM

• Nice to see that the generics fanatics try to make generic code look as shitty as C++ template code by insisting on there beloved, brain dead single letters . Good work. I am sure once you finally manage to get operator overloading bolted onto the language we can expect the full "beauty" of C++ code.

  As for wasting time on adding generics to every Swing component which can't run away fast enough: Why not first fix the stinkers in Swing instead of wasting time with generics and adding new and improved bug?. Things to fix are for example:

  - JTable: broken since day one and counting (broken selection model (the work of a "genious"), broken cell "fit to width" calculation, edit cancelation happening or not happening at strange times, DnD support a joke, focus handling interaction with the scroll pane a desaster, etc., etc., etc.)

  - JFileChooser: So sub-standard many developers still use the AWT file choose in their Swing programs

  - The whole stinking text package with the barely working HTML parsing, the joke passing of as RTF support, the still not correctly working styled text (superscript/subscript anyone?), the deviation from Swing's normal Model-View architecture with the introduction of documents, etc.

  - Still transparency for top-level windows doesn't work, still no support for non-rectangular windows, still no way to provide a frame with a correctly sized icon, still no keyboard-layout independent support for Robot, still standard dialogs like a font chooser missing, etc.

  So if you are bored, don't start playing with generics. Fix the existing problems. After that, finally adapt Swing to the "new", post Java 1.1 collection classes. E.g. JTree constructors still live in the world of Hashtable and Vector and have't heared of Map and List.

  And after you have done the above homework, go to your bug parade, and fix every Swing and AWT bug you have there. It is not as if there is a shortage of bugs. After that, do ... nothing! At least nothing with the Swing code. Fix the documentation. There are so many mystery methods and classes in Swing and AWT, you could cut a few episodes of the X-Files out of it.

  Oh, I know it won't happen. It hasn't happen in the last nine years people have waited for Sun to get their GUI stuff in order.

  Posted by: ewin on August 27, 2005 at 11:15 PM

• For me, the new filtering and sorting API will be one of the most useful upcoming additions to Swing. Since the collections framework already benefits tremendously from generics (IMO), generifying the Swing API where appropriate makes perfect sense.

  Though I agree that Swing does have some quirks that should continue to be addressed, any criticism that is based solely on generics making things more difficult to read seems to confuse to concept of syntax with that of semantics. A little more syntactic clutter (if you think of it like that) is a fair trade-off for semantic cleanliness, IMO. Once you internalize the syntax, it ceases to be a source of confusion and becomes valuable information about a program’s behavior. The elimination of casting – which is semantic clutter – is an improvement that ultimately makes code easier to understand and maintain. In other words, “easy to read” does not always equal “easy to understand”.
  

  Posted by: ajconover on August 28, 2005 at 10:45 AM

• Anyone that sacrifices readability for type safety deserves neither.

  Posted by: rickcarson on August 28, 2005 at 08:17 PM

• -1 on generification. Don't add generics to Swing... Please...

  Posted by: vprise on August 29, 2005 at 06:28 AM

• +1 for me.

  about the Row problem, why not creating the array
  using the complete notation with a wildcard, i.e :

  instead of
  Row[] array=new Row[15]

  i think you should write
  DefaultRowSorter.Row[] array=
  new DefaultRowSorter.Row[15];

  Rémi Forax

  Posted by: forax on August 29, 2005 at 01:17 PM

• oups, sorry but my comment was not correctly
  formatted and the end is missing.

  So about the Row problem,
  instead of writing :

  Row[] array=new Row[15];

  you could write :

  DefaultRowSorter.Row[] array=
  new DefaultRowSorter.Row[15];

  
  Rémi Forax

  Posted by: forax on August 29, 2005 at 01:23 PM

• oups, sorry but my comment was not correctly
  formatted and the end is missing.

  So about the Row problem,
  instead of writing :

  Row[] array=new Row[15];

  you could write :

  DefaultRowSorter<?>.Row[] array=
  new DefaultRowSorter<?>.Row[15];

  
  Rémi Forax

  Posted by: forax on August 29, 2005 at 01:26 PM

• Hi Remi,
  That doesn't appear to work. I tried this in a sample class:

  public class Foo<T> {
  private T t;
  private Foo<?>.Inner[] inner;
  public Foo(T t) {
  this.t = t;
  inner = new Foo<?>.Inner[10];
  }
  private class Inner {
  }
  }

  Foo.java:11: '(' or '[' expected
  inner = new Foo<?>.Inner[10];

     -Scott

  Posted by: zixle on August 29, 2005 at 02:27 PM

• ewin,
  We are aware of many of the problems you side and are actively working on them.
  "JTable: broken since day one": Do you have specifics?
  "DnD support a joke": Shannon blogged on this recently. If you are anxious download the mustang bigs and have a look.
  "focus handling interaction with the scroll pane":Don't know about this.
  "JFileChooser": A ton of work has gone on here. Have you tried the latest?
  "Text Package": We've focused a lot on stability and bugs at the expense of new features. The core text package is in the best shape it has ever been in.
  "Still transparency for top-level windows doesn't work": Yes, this is annoying to say the least. Supports translucent window wreaks havoc with hardware acceleration. Hopefully we will have a good answer for this for 1.7.
     -Scott
  

  Posted by: zixle on August 29, 2005 at 02:36 PM

• oups, it works with eclipse but it doesn't work with javac.

  I don't know if it's legal or if its an eclipse bug
  but this syntax seems really what's you need :)

  Note that it works if i replace all wilcards by its raw type :

  public class Foo {
  private T t;
  private Foo.Inner[] inner;
  public Foo(T t) {
  this.t = t;
  inner = new Foo.Inner[10];
  }
  private class Inner {
  }
  }

  perhaps you could ask to Peter von der Ahe.

  Rémi

  
  

  Posted by: forax on August 30, 2005 at 06:10 AM

• +1 on generics
  +1 on GTXXXX instead of "M". Naming something "M" demonstrates a disrespect for your API readers and willingness to follow a herd into a ditch. Generics needs to quickly rid itself of its "genius only" mystique. It's not good to present an API that most find unapproachable. How much more approchable does the long type name make it? Very!

  Posted by: michaelbushe on August 30, 2005 at 11:18 AM

• Remi,
  Arrays containing generified types just don't work, it's a bug in Eclipse that it works.
     -Scott
  

  Posted by: zixle on August 31, 2005 at 04:18 PM

• Arrays of inner classes are a pain:

  
  public class Foo<T> {
  private T t;
  private Foo.Inner[] inner;
  public Foo(T t) {
  this.t = t;
  inner = new Foo.Inner[10];
  Inner i = inner[0]; // unchecked warning
  }
  private class Inner {
  }
  }

  
  If you change the declartion of the array to

  Foo<?>.Inner[]

  the warning turns into an error.

  However, I'm not sure yet if javac or Eclipse is
  right. Eclipse seems to have a stronger case.
  

  Posted by: peterahe on August 31, 2005 at 10:49 PM

• I have open a bug to track this problem :
  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6318240
  else
  If you use wilcards at every place there is no problem :

  public class Foo {
  private T t;
  private Foo.Inner[] inner;
  public Foo(T t) {
  this.t = t;
  inner = new Foo.Inner[10];
  Inner i = inner[0]; // error
  Foo.Inner i=inner[0] // ok
  }
  private class Inner {
  }
  }

  
  Scott, I've replaced in DefaultRowSorter all reference to Row
  by DefaultRowSorter.Row and it seems
  there is no problem (with eclipse).

  I can see an alternative solution, split the DefaultRowSorter
  in two classes. Create a static class named by example
  Internal. This class contains all fields needed by Row.
  (modelIndex,cachedSortKeys,useToString,sortComparators)

  Makes Row an inner class of this one.
  Create a field internal in DefaultRowSorter and delegate
  all calls to these fields in DefaultRowSorter by delegating
  to internal.

  class DefaultRowSorter<M> {
  static class Internal {
  int modelIndex;
  boolean useToString; // etc...

  class Row {

  } // not parametrized by M.
  }
  private final Internal internal=new Internal();
  public int convertRowIndexToModel(int index) {
  if (internal.viewToModel == null) {
  if (index = getModel().getRowCount()) {
  throw new IndexOutOfBoundsException("Invalid index");
  }
  return index;
  }
  return internal.viewToModel[index].modelIndex;
  }
  ...
  }

  
  Rémi

  Posted by: forax on September 01, 2005 at 07:25 AM

• Rémi,
  Interesting idea. I ended up making Row static and having a field for the DefaultRowSorter. That way I can still use Row in an array and don't have to worry about all this;)
     -Scott
  

  Posted by: zixle on September 01, 2005 at 02:07 PM

• Slightly OT: [quote]"JFileChooser": A ton of work has gone on here. Have you tried the latest?[/quote]
  I think one of the main issues here is look and feel fidelity... on Windows XP it is quite bad and users notice it right away and complain A LOT.

  e.g.:
  - It can be very slow (simply constructing a JFileChooser takes many seconds sometimes, you don't have to show it.)
  - The context menus available in the native file chooser are not available.
  - The view selection (details, list, thumbnails, etc.) is only partially supported (list and details) and uses buttons instead of the drop down menu of the native component.
  - The table headings in the details view look completely different when they are being repositioned.. the native L&F shows a translucent heading being dragged and a vertical bar indicating where the heading will be inserted when dropped.
  - The details view does not allow SORTing by clicking the column heading... this one missing feature makes the chooser very inconvenient for users that work that way e.g. sorting by date to find their file in a list of hundreds. And it just so happens that this is somewhat relevant to the topic of this blog.

  I realize it may not be possible given various constraints that you are under but I urge you to consider fixing the JFileChooser Win XP L&F in Java 5.. waiting 3/4 of a year for Mustang for these sorts of fixes is killing us.

  At the very least - Fix it COMPLETELY for Java 6.. we can't afford to wait for Dolphin.

  Posted by: swpalmer on September 12, 2005 at 07:37 AM

• Hi,

  A little off topic, but here goes...

  Are table filters (especially regex) going to be able to filter off of RENDERED values, or just underlying model values? I often use custom renderers to display text that is NOT straight from the underlying model's object.

  Thanks!

  Posted by: yonashi on February 16, 2006 at 09:25 PM

• yonashi,

  Are table filters (especially regex) going to be able to filter off of RENDERED values, or just underlying model values?

  Filtering is only done at the model level. It's possible to supply a different TableModel to the RowSorter that produces different values, but that would be tricky...

      -Scott

  Posted by: zixle on February 21, 2006 at 10:28 AM

• a little bit off topic, but....

  i got table sorting working sweet, in the win xp look and feel the jtable column header are highlighted when the mouse hovers over them, giving the user the impression they can click on them to sort the column.

  some of my tables are not suitable for sorting and as such i would like to turn the hover effect off. any ideas on how i can do that ?

  Posted by: bumperbox on February 21, 2006 at 11:03 PM

• bumperbox,

  some of my tables are not suitable for sorting and as such i would like to turn the hover effect off. any ideas on how i can do that ?

  Good question. We currently don't have a way to do that. I'll add that to my list of things to do.

  Thanks for the feedback!

      -Scott

  Posted by: zixle on February 22, 2006 at 10:30 AM

• Ive just tested sorting with Mustang 1.6.0-beta2 and im a little disappointed. Its slow. I have a table with about 10 000 rows. Sorting takes about 3 sec. My old custom sorting sorts the table without any dalay.

  Posted by: waldekp on August 18, 2006 at 12:33 AM

• waldekp,

  I have a table with about 10 000 rows. Sorting takes about 3 sec.

  How were you doing the comparisons? The sorting doesn't cache the values from the model, and it uses a Collator. Caching could undoubtedly improve the performance at the cost of a lot more memory.

  Would you happen to have a test case I can use, as well as letting me know if you were using Collator?

  Thanks,

      -Scott

  Posted by: zixle on August 18, 2006 at 11:05 AM

• You're right. I didnt use collator. Ive tried sorting with collator and its the same daley - about 3sec.

  Posted by: waldekp on August 20, 2006 at 10:26 PM

• waldekp,
  You're right. I didnt use collator. Ive tried sorting with collator and its the same daley - about 3sec.

  While Collator is to be used for locale sensitive string comparisons, Swing's RowSorter does not require it's usage. If you want, you can specify a Comparator that uses a different algorithmn, such as String's compareTo.

      -Scott

  Posted by: zixle on August 21, 2006 at 08:16 AM

This work is licensed under a Creative Commons License.