Ethan Nicholas's Blog

Mnemonic Magic

Swing's mnemonic system is based around two properties: mnemonic (or displayedMnemonic) and displayedMnemonicIndex. They're powerful enough to do everything you need, but then again, so is machine code. There are a number of problems and limitations with the skeletal mnemonic support built in to Swing.

The Problems

• They're annoying to define - One common approach is to use a properties file to define actions. You generally end up with a property for the action's name, a property for its mnemonic, and an (optional) property for its mnemonic index -- three properties for the name of each action.
• It's easy to accidentally duplicate mnemonics - if you add new actions, or build a new menu from existing actions, it's incredibly easy to forget that (say) both "Lock" and "Layout" are using 'L' as their mnemonic character
• It's difficult to spot mistakes - the only visual cue to a mnemonic is an underline. Using Windows XP with default settings, the underline doesn't even show up unless you hold the Alt key. Mistakes are easily overlooked, and there is no error reporting.
• The same action may need different mnemonics in different contexts - If an action appears (say) in both a menubar menu and a right-click context menu, a mnemonic character which works in one menu may conflict in the second menu. I once ran into a menu item with a short name which, in a particular context menu, conflicted with other menu items on every single character in its name. With the action defined by static properties, though, you have to use the same mnemonic everywhere... so what do you do? Not have a mnemonic anywhere just because it conflicts in one rarely accessed location?
• It poses serious problems for internationalization - A quick survey of the bug database turned up a number of bugs relating to incorrect translations of mnemonics and mnemonic indices: 6411189, 4983399, 4664265, 4735183. There are two general problems: first, all of these properties are interrelated, and it's easy to miss one of them during translation -- leading to incorrect mnemonics or (worse yet) a mismatch between the actual mnemonic and the underlined character. Second, translation is quite complicated because the translator has to check that the mnemonic doesn't conflict with any other mnemonics in every menu in which the action appears. If the folks at Sun are making these kinds of mistakes, what hope is there for the rest of us?

Because of these annoyances, the unfortunate fact is that most Swing programs (at least among the ones I've seen) don't actually use mnemonics at all. That's sad, especially when you consider that other toolkits don't seem to share this aversion for mnemonics.

The Solution

The solution I propose is twofold. First, combine all three properties into a single value. Second, eliminate the need to specify mnemonics at all in most cases.

Combining all three properties is quite easy. You just need to embed the mnemonic and mnemonicIndex information into the label itself.

Before:

action.saveAs.name          = Save As...
action.saveAs.mnemonic      = A
action.saveAs.mnemonicIndex = 5

After:

action.saveAs.name = Save _A_s...

I chose to use a pair of underscores surrounding the mnemonic character to identify it, as opposed to most toolkits which use a single escape character. Windows, for example, would have specified the label as "Save &As...". The advantage of the underscore encoding is that there is essentially no chance of ever "accidentally" specifying a mnemonic character, whereas you might legitimately try to use an ampersand in a control, which in Windows would lead to an incorrect mnemonic.

Being able to specify the mnemonic character in this fashion carries a lot of benefits. Because it's a single value, specifying it in properties files is much easier, internationalization is much easier, and it's impossible to accidentally have a mismatch between the index and the mnemonic character. While this is much easier to deal with than three separate properties, it doesn't solve some of the other thorny problems, like trying to prevent duplicate mnemonics.

The rest of the solution is automatically assigning mnemonics. This is easier than it sounds, because we tend to follow certain heuristics when assigning mnemonics to actions and labels, and we can just turn those heuristics into a program. The algorithm I use is as follows:

1. Try the first character in the string. Check to see if it is available (among other menu items in the menu or components in the window, as appropriate).
2. If that isn't available, try capitalized letters in the order in which they appear.
3. If no capitalized letters are available, try lowercase consonants.
4. If no lowercase consonants are available, try lowercase vowels.
5. If no lowercase vowels are available, give up.

So my framework automatically assigns mnemonics to all actions (and specified components) based on these rules. 99% of the time it picks the same mnemonic you yourself would have. If it doesn't do what you want, you can manually specify it using a pair of underscores around the character in question. You can also force it not to choose a mnemonic at all by putting two underscores in a row at the end of the string. This makes translators' lives far easier, because they can generally just not worry about mnemonics and let the program "do the right thing". Furthermore, my framework has automatic error checking and will, if assertions are enabled, throw an AssertionError when a menu or dialog contains duplicate mnemonic characters.

Unfortunately...

There are some other details, of course, like the API calls necessary to actually work with this framework, but they aren't relevant because I can't release the code to the public. It's part of a proprietary application which my employer owns the rights to, so you'll just have to use your imagination. Fortunately the code itself is relatively straightforward -- you could easily duplicate this setup if you wanted to, and probably save yourself a lot of time overall (if you're working on a big application). There is hope, however...

JSR 296

JSR 296, "Swing Application Framework", aims to (among other things) define a standard way of representing and managing Actions. My hope is that, as part of this, JSR 296 will include a better means of managing mnemonics than having to specify three properties individually. At the very least I'd like to see the mnemonic be able to be embedded directly into the string (as in "Save _A_s...", "Save &As...", or something equivalent), but trust me when I say that the automatic mnemonic management is incredibly nice also.

Considering that I have a seat on the JSR 296 expert group, I will definitely be pushing for some sort of solution to these issues. I haven't actually discussed this with anyone yet, though, so maybe they already have an even better solution in mind (yay!) or have decided that this issue isn't worth pursuing (boo!). It's also entirely possible that I'm the only weirdo that considers this a significant issue in the first place.

JAXX

If you've used my JAXX framework, you may be a little surprised about this entry, considering that JAXX doesn't do automatic mnemonic management or allow you to embed mnemonic markers directly into the label. That was a deliberate, albeit painful, decision, and it comes back to JSR 296. Given that there's eventually going to be a standardized Swing application framework, I'm taking a wait-and-see attitute, carefully avoiding doing anything in JAXX which might conflict with the Swing framework-to-be. If JSR 296 does define a standard way to identify mnemonics, I'll use the same syntax. And if it doesn't, I'll do it my way. Regardless of the outcome, I expect that JAXX will eventually support this sort of mnemonic magic.

What do you think?

So, what do you think? Do you agree that Swing's approach to mnemonics could use some improvements? Want automatically managed mnemonics? Hate the idea? Like being able to specify mnemonics directly in the string, but don't like the syntax? Or am I just altogether crazy?

Comments

• A more common convention (used in various platforms and frameworks) involves prefixing the mnemonic character with the & symbol (double it for a literal ampersand), instead of preceding/trailing underscores.
  I second your concerns about having multiple properties for a single UI element. See my comments on this issue on Hans Muller's blog.
  - Chris

  Posted by: chris_e_brown on June 06, 2006 at 04:46 AM

• 
  I agree with this nice approach, Nicholas, and did something similar on previous project. I think it's a must because the mnemonic is linked to the translation. For instance, when translating Save into another language, the S mnemonic doesn't make sense any more,
  and should be translated together with the label. But maybe translators (and their tools) would prefer to specify them separately rather than embed such indicators into the translated strings, i don't know.

  I used a preceeding & in past project, to specify the mnemonic. Problem was that the project was a mix of older Netbeans designer GUIs, and new ones using a minimal framework i threw together. The framework used mnemonics in the resource labels. But the Netbeans designer doesn't of course. So it was a bit inconsistent.

  In general I think it's best to be specific, ie. to have to specify the mnemonic explicitly rather than guessing anything automatically. But maybe i'm wrong.

  
  

  Posted by: evanx on June 06, 2006 at 05:29 AM

• Chris - I'm well aware of the ampersand convention, and I touch on my reasons for bucking the trend in this entry. Keep in mind that this was something I did for a two-person project; for a wider audience (all Swing developers, say...) it might make more sense to follow the herd. Or it might not (I personally view ampersand as an unfortunate choice). Did you just want to make sure I was aware, or do you prefer ampersands?

  I also posted a response over on Hans' blog.

  Posted by: enicholas on June 06, 2006 at 05:57 AM

• I second the above proposal for & to specify a mnemonic. No reason to create just another convention.

  I am against "automatically assigning mnemonics" it, since
  it brings no real value and because you would give an excuse to the developer/designer to forget about UI conventions and useablity.

  
  There are standard(locale dependent) mnemonics for menu-items like "New, Save, Print, Cut, Copy, ... " that should be the same for every application regardles of their position.
  Adding a new menu-item could change many mnemonics on which a existing user was used to.
  How many *real* menu items does an application have. For an application with around 50 items chosing the right mnemonics, takes very very litle time compared with the time to develop the application.

  A professional application always ships with correct mnemonics.

  
  

  Posted by: tiom on June 06, 2006 at 06:04 AM

• tiom: In practice, none of your concerns have turned out to be issues in our application. Keep in mind that you can manually specify mnemonics to handle edge cases -- so we have one or two menu items which had to have manually-specified mnemonics, and every other one ended up correct with no special effort. If a new menu item would change familiar mnemonics, we just manually specify a non-conflicting mnemonic for it. It really isn't a big deal.

  Also, it is waaaay more than fifty mnemonics in a big application. Many hundreds is much more typical. It's not just the top-level menus -- it's every context menu (of which there can be dozens) and every form element label in every dialog box. Saying that "A professional application always ships with correct mnemonics" is equivalent to saying "A professional application always ships without bugs" -- it's a nice theory, but is clearly incorrect in practice. I have seen duplicate or missing mnemonics in a number of flagship applications from major software manufacturers.

  Posted by: enicholas on June 06, 2006 at 06:28 AM

• I have to say I like the usage of underscores over ampersands if for no other reason than it's easier to embed in XML. Entity escaping can get to be a pain.

  Posted by: joshy on June 06, 2006 at 08:04 AM

• I believe Hans had something for this in JSR 296 -- at least I know we talked about it. As it turns out, the "&" tradition has gotten the boot over at Microsoft. In Avalon they use an "_" such as: "Save _As". In addition to looking better (hey, it even looks like a mnemonic!), it also works well in XML. Which, if you have XAML to define your UI in, is a big deal.
  Richard

  Posted by: rbair on June 06, 2006 at 08:32 AM

• All of the arguments for underscores certainly are more convincing than sticking with the ampersand convention. I'd back it having read the other comments.

  - Chris

  Posted by: chris_e_brown on June 06, 2006 at 09:58 AM

• If text resources will be put into XML files, the underscore makes much more sense than the ampersand and for consistency with properties i would go with the underscore. In my use case i used the ampersand, because our translators were used to them and all localizable text was outside XML files.

  About my concerns. The algorithem(which is wrong for at least one language) for automatic mnemonic assigment would not work for practicly all every-day applications I use. ie. in Firefox the mnenomic for "Save As" would be 'S' and not the second 'a', in OpenOffice, the "Properties" would get the mnemonic 'P' instead of "Print", ... .
  So I would have to manual correct them and because I am not smart enough to forsee what the algorithem would do other mnemonics I would recheck the entire menu. So if I apply the algorithem to applications that I develop it would not relieve me from the burden of mnemmonics. There would be no duplicates but there is no guarantee that the mnemonics are correct(common items with common mnemonics).
  It would be far more heplfull if the framework when started with a special flag, would highlight items with conflicting and missing mnemonics with a red color, so errors can be quickly spoted and corrected.

  Now consider that i would used the algorithem, correct the mnemonics so common items would keep their common mnemonics. I would still need to
  explain to the translators why some string have mnemonics, some not and why they have to insert a mnemonic for a string that does not have a mnemonic
  char in the original.

  
  A professional application can ship with correct mnemonics but it cannot ship without a bug. see notepad for example

  Posted by: tiom on June 06, 2006 at 10:43 AM

• 
  I also proposed using the "&" convention to mark mnemonics in labels
  in the JSR 296 (app framework) session at JavaOne. For example, see slide 42:
  http://weblogs.java.net/blog/hansmuller/archive/ts-3399-final.pdf

  
  
  Not exactly a novelty :-). Automatically assigning mnemonics sounds
  like a reasonable idea, I think I'd be happy just overriding the
  menemonics that the automatic algorithm got "wrong". I imagine there
  wouldn't be very many. After an application shipped, one would have
  to guard the stability of mnemonic assignments, you wouldn't want
  a new menu item could grab the mnemonic for an item people had been
  using for a while.

  I don't think one could extend this idea to
  shortcuts or tooltips or any of the other action attributes developers
  have to keep straight.

  Posted by: hansmuller on June 06, 2006 at 11:14 AM

• joshy -- Thanks for the reminder on ampersands in XML. That was actually the main reason I originally avoided ampersands (since Swing XML layouts are near-and-dear to me) but it had slipped my mind while writing this.

  Hans -- I'm very glad to see you already have plans for this, but I'm going to have to push for underscores rather than ampersands due to the ease-of-XML-encoding issue. Even if the JSR 296 framework doesn't use XML for specifying action names, there will definitely be a lot of people (myself included) who need to encode the information using XML for various reasons.

  Posted by: enicholas on June 06, 2006 at 11:30 AM

• I agree that mnemonics would better be specified in strings. I'm also not fussed about which character should be used, but if HTML were more widely supported in Swing, I'd suggest the mnemonic was just specified by using the <u> tag.

  I side with tiom about automatically assigning mnemonics. I don't see how his concerns could have not turned up for you in practice, assuming you're using mnemonics for field labels and not just buttons and menus. 50 mnemonics is a conservative example in my view; any fairly complicated dialogue, with tabs, buttons, and an active menu bar, would easily have 20 just for that one dialogue. That's already most of the alphabet, so just by adding a component (or removing one) in a later release, the automatically-assigned mnemonics for another component would probably change. Adding a new menu would potentially change the mnemonics in all the dialogues! You're then either faced with the puzzle of finding the mnemonic you need to assign manually to get all the other mnemonics back to whatever they were, or you have to deal with unhappy users, especially ones with accessability issues, who have to get used to a new set of mnemonics.

  One additional concern that tiom didn't mention is that not all languages have such a concept as vowels and consonants. Admittedly, in most of those languages, the whole concept of mnemonics is foreign; Chinese translations of applications, for example, still use English letters for mnemonics.

  Thanks for bringing this up—I'm glad I'm not the only one who takes mnemonics seriously. :)

  Posted by: mscheper on June 06, 2006 at 05:34 PM

• 
  Shouldn't there just be an easier way to retrofit the abstractAction class?

  
  So using this action I pulled from my Swing code:

  
  public class SaveAsFileAction extends AbstractAction {

  private static final String CAPTION = "Save As...";
  private static final String TOOL_TIP = "Save with a new name";
  private static final String STATUS_BAR = "Saves the file with a new name";

  
  public SaveAsFileAction() {

  putValue(DEFAULT, CAPTION);
  putValue(NAME, CAPTION);
  putValue(SHORT_DESCRIPTION, TOOL_TIP);
  putValue(LONG_DESCRIPTION, STATUS_BAR);
  putValue(SMALL_ICON, IconLoader.getIcon("SaveAs16.gif"));
  putValue(ACCELERATOR_KEY,
  KeyStroke.getKeyStroke(KeyEvent.VK_A, Event.CTRL_MASK));
  putValue(MNEMONIC_KEY, new Integer('A'));
  }

  AbstractActions really rock from my perspective but if someone thinks there is a better way I'm open to hear it. The code snippet above is from TFrame which is a very old blue collar Swing framework.
  

  Posted by: cupofjoe on June 06, 2006 at 06:05 PM

• 
  So looking at the interface that describes the Action variables used for the map collection I see this:

  public interface Action extends ActionListener {
  /**
  * Useful constants that can be used as the storage-retrieval key
  * when setting or getting one of this object's properties (text
  * or icon).
  */
  ===extra code removed here======
  ...
  ===extra code removed here======
  /**
  * The key used for storing a KeyEvent to be used as
  * the mnemonic for the action.
  *
  * @since 1.3
  */
  public static final String MNEMONIC_KEY="MnemonicKey";

  
  So maybe I'm thinking too much like a hacker from my days working in telecom but if it were me I would add a MNEMONIC_COUNT item above. Set it to 1 as a default and put it in the map. (Base 1 NOT zero).

  
  Then I would just tell everyone to set the map to a different value but it defaults to '1' which is the first 'char' found. Then pull the JComponent code from there. Backward compatiable and hopefully I haven't run out of duct tape or super glue yet. ;-)

  
  I'm a big fan of AbstractAction because I use it in a LOT of my code and it makes things really easy IMHO.

  Posted by: cupofjoe on June 06, 2006 at 06:21 PM

• I would split the difference between automated assignment and manual assignment--I think IDEs should both have an "assign", "check assignments", "reassign" options for menu items. Ideally, menu configuration should be set up in such a way that it is parseable and thus testable from a dev environment. Even having a small Swing application do this for you (read a properties file, instantiate the menu, check assignments) is not out of the question.

  A way to summarise some of the other comments is that the algorithm needs to start by comparing the menu item text against a "standard" list (in that locale) with "standard" assignments.

  Regards
  Patrick

  Posted by: pdoubleya on June 06, 2006 at 06:36 PM

• Definitely agree that incorporating the mnemonic character in the label is preferable and think it would be appropriate for JSR 296.

  What character? My suggestion would be to go with the flow. Most folks seem to use ampersand, most likely after following Microsoft's convention. If Richard is correct and Microsoft has indeed switched to using a leading underscore, and folks are jumping on board, then we should seriously look at that. In any case, a survey of precedents is in order. I don't like the double underscore for two reasons: no one else does it (OK, I'm guessing there ;-), and it is really hard (for me) to read and decipher. Sorry. The single ampersand or underscore is light-years more readable.

  I've thought about the automatic mnemonic heuristic too, but it suffers from instability. It might still have use for fields and buttons, but I don't think you'd want to use it for menu items. I really like tiom's idea about a mnemonic lint for flagging dups.

  -Bill

  Posted by: wohler on June 12, 2006 at 03:01 PM

• Others have said it as well, just adding my bit to let you know automatic assignment of mnemonics is a really bad idea. I've worked for a company that tried it and when it at first didn't work tried it some more and in the end they just gave up.

  Automatic assignment almost never assigns the letters that you somehow through experience with other programs would expect. Also the differences between languages means basically you also need different assignment rules, but in the end the stickler is that you do NOT want your mnemonics to change EVER. Well at least not without a very good reason and even then you have to really think about it and be prepared to retrain your user base, because a user that is finally used to pressing A for Aquisitions doesn't want to relearn the application when support for Accounts is added.

  Posted by: quintesse on June 29, 2006 at 01:46 AM

• How would the escape code work for non-English keys? For example, let's say the menu option is "Pr_éférences" indicating the 'é' key is the shortcut. I believe this is in the same spot as the 7 key on an English keyboard. Because the JMenuItem.setMnemonic(char mnemonic) is deprecated (only works for 'a'...'z'), you are supposed to use JMenuItem.setMnemonic(int mnemonic) and supply a VK_XXXX constant. So you need a class that will translate 'é' from the French keyboard into KeyCode.VK_7 to properly setup the mnemonic. Is there such a class?
   
  P.S. Why not use some other rarely-used char like '~' or '|' as the escape code, since underscore is sometimes hard to pick out? My vote is against the ampersand because of XML. If I had to vote for an escape char I'd pick the tilde '~'.

  Posted by: kmenningen on December 18, 2006 at 01:07 PM

• Ethan, I like the heuristics you suggest.

  I didn't see a link to an implementation, so here's my own: http://grinder.svn.sourceforge.net/viewvc/grinder/trunk/source/src/net/grinder/console/swingui/MnemonicHeuristics.java?view=markup

  To use, simply construct an instance of MnemonicHeuristics around a particular container (e.g. JMenu, JMenuBar, ...), and it will do the rest - automatically setting and updating mnemonics for all AbstractButtons belonging to that container. I didn't do the '&', '_', ... bit, but the MnemonicHeuritistics class will respect explicit mnemonics that are set before it is called.

  Regards,

  - Phil
  

  Posted by: philipa on December 25, 2006 at 01:33 AM

• I've updated my implementation to support explicit specification of mnemonics with an underscore.

  With regard to kmenningen's question, I do think there's an I18N problem with mnemonics in general since
  JMenuItem.setMnemonic(int mnemonic)
  only takes key codes. I've mapped from characters to key codes with
  something simple like:

  final char upper = Character.toUpperCase(c);
  if (KeyEvent.getKeyText(upper).equals(String.valueOf(upper))) {
  // upper is a valid mnemonic.
  }

  
  There may be a more I18N friendly way of mapping from characters to key codes, but its not obvious to me. I welcome suggestions.

  Posted by: philipa on December 27, 2006 at 08:36 AM

This work is licensed under a Creative Commons License.