The Visual Library is one of the coolest things that the NetBeans guys delivered with NetBeans 6. It is a rich API which allows you to create a sort of "blackboard" where objects can be added, removed, edited, moved, resized, and connected in a visual graph. The cream on the cake is that you can use the Visual Library even in regular Swing applications by just adding the related JAR to the classpath, since it has no dependencies on the NetBeans core.

The Visual Library is deeply used inside NetBeans: it's at the core of Matisse, the visual designer, as well as the UML support, the J2ME designer, in a few words everything that renders a graph. When Roumen posted a screencast about using the Visual Library I got really excited since I understood that it would have allowed me to easily implement a new feature in blueMarine: the (Virtual) Light Table. It is a visual component where you can drop thumbnails from your photo collection and arrange them as you prefer with the mouse. You can see a screenshot below and a very short screencast (.MOV, 300k) is also available.




Now I'm going to tell you how I was able to implement the LightTable in a matter of hours (tests excluded) yesterday night with the Visual Library.

WARNING: this blog assumes you're confident with the basic concepts of Swing and the DataSystem and Nodes API of NetBeans.

You're going to see all the relevant code in the sketches below; in any case, the sources can be checked out with Subversion from https://bluemarine-incubator.dev.java.net/svn/bluemarine-incubator/trunk...

The binary code isn't available in blueMarine yet (hey, I just wrote it last night! Give me the time to test it... ;-) - it will be available soon by means of the Update Centers and will be part of the next round of blueMarine demos I'll give starting from the next month.


Some blueMarine APIs

Of course I've been facilitated by the consolidation of the blueMarine APIs, that are a set of NetBeans modules for manipulating photos (and - with the latest refactoring - any kind of media type). Since I'd like to focus on the Visual Library and not on blueMarine, I'm just telling you what the few blueMarine classes you'll see in the code listings are about:

• Thumbnail is just a reduced size image used as a photo preview. It is smart enough to be asynchronous, that is it can be generated in background and fire events when data (raster and metadata) are available.
  
• ThumbnailManager is the manager for Thumbnails; you usually call it as
  

  DataObject dataObject = ...;
  Thumbnail thumbnail = thumbnailManager.findThumbnail(dataObject);
  

  Remember that DataObject is the standard way of representing a datum with NetBeans. Whenever you're dealing with a photo, the code above will give you a working thumbnail (or create one if it doesn't exist).
• ThumbnailRenderer is the renderer for thumbnails. It is able to deal with the asynchronous model of a Thumbnail, that is to quickly render a placeholder if the Thumbnail is not ready yet, and repaint it later when the image is available.
• ThumbnailTracker is a container of a set of Nodes which are capable of update themselves when the related Thumbnails are updated. The typical use is:
  
  Node node = thumbnailTracker.add(dataObject.getNodeDelegate());
  

For the record, these APIs are part of the blueMarine Core, the foundation APIs of blueMarine. They are going to be frozen an documented soon. Stuff for another blog post.


The Scene

The Scene is the container for the "blackboard", that is the container of all the graphics objects (that are called "Widgets"). While Roumen's screencast introduced the regular Scene class, I'm going to use an ObjectScene, which provides some additional features:

• it defines the concept of "selected" widgets, as well as "hovering" widgets (i.e. the object under the mouse) and others; and fires the proper events to notify external listeners;
• it provides an association between a Widget and a custom object - that is, the model for your Widget.

For instance, the following code will create a working ObjectScene with the capability of add and remove DataObjects from it:

    /** This object manages a set of thumbnailed nodes. */
    private final ThumbnailTracker thumbnailTracker = new ThumbnailTracker();
    
    /** The scene manager from Visual Library. */
    private final ObjectScene scene = new ObjectScene();
    
    /** The main layer where Widgets are added. */
    private final LayerWidget mainLayer = new LayerWidget(scene);

    private void internalAdd (final DataObject dataObject, final Point location)
      {
        final Node node = thumbnailTracker.add(dataObject.getNodeDelegate());
        final Thumbnail thumbnail = ThumbnailManager.Locator.findThumbnailManager().findThumbnail(dataObject);
        final ThumbnailWidget widget = new ThumbnailWidget(scene, thumbnail, node);

        // widget.setPreferredLocation(widget.convertLocalToScene(location));
        final Point sceneLocation = scene.convertViewToScene(viewLocation);
        final Point localLocation = mainLayer.convertSceneToLocal(sceneLocation);
        widget.setPreferredLocation (localLocation);

        mainLayer.addChild(widget);
        scene.addObject(dataObject, widget);
        widget.getActions().addAction(scene.createSelectAction());
        widget.getActions().addAction(scene.createObjectHoverAction());
        scene.validate(); // See first entry of http://graph.netbeans.org/faq.html
      }

    private void internalRemove (final DataObject dataObject)
      {
        scene.removeObject(dataObject);
        final List widgets = scene.findWidgets(dataObject);
        scene.removeObject(dataObject);

        for (final Widget widget : widgets) // removeObject() doesn't remove widgets
          {

            widget.removeFromParent();
          }
      }

Just a few comments:

• ThumbnailWidget is a specialized visual object that I'll describe below.
• A Scene can contain many layers (LayerWidget), that are the place where Widgets are placed. In the simplest cases, you just need one of them.
• Capabilities can be added to Widgets by adding WidgetActions to them. Most of WidgetActions can be created from an ActionFactory (see code sketches below), while some are provided by the ObjectScene (in the code, the action for managing the selection and the hovering status).
  

Once you have created an ObjectScene, you just get the Swing components out of it:

    /** The view component. */
    private final JComponent view = scene.createView();
    
    /** The view component. */
    private final JComponent satelliteView = scene.createSatelliteView();

The view is the blackboard renderer itself, and you just need to place it in a Swing JWindow or such (in most cases you'd put it into a JScrollPane first); the satelliteView is an optional component that provides a "bird's eye" view of the blackboard.

As those components are just Swing components, you can integrate them with things such as drag and drop. For instance, the following code enables dropping DataObjects directly on the LightTable:

    private final DropTarget dropTarget = new DropTarget()
      {
        @Override
        public void drop (final DropTargetDropEvent event) 
          {
            try
              {
                for (final DataFlavor flavor : event.getCurrentDataFlavors())
                  {
                    final Class<?> clazz = flavor.getRepresentationClass();

                    if (DataObject.class.isAssignableFrom(clazz))
                      {
                        final DataObject dataObject = (DataObject)event.getTransferable().getTransferData(flavor);
                        //
                        // This tests if the current DataObject has Thumbnail capability.
                        //
                        if (dataObject.getLookup().lookup(Thumbnail.DataProvider.class) != null)
                          {
                            event.acceptDrop(event.getDropAction());
                            internalAdd(dataObject, event.getLocation());
                            event.dropComplete(true);
                            return;
                          }
                      }
                  }
                
              }
            catch (Exception e)
              {
                logger.throwing(CLASS, "drop()", e);
              }

            event.dropComplete(false);
          }
      };

...
    view.setDropTarget(dropTarget);

Creating a Widget

The Visual Library comes with a lot of pre-made Widgets, such as LabelWidget (which contains a text), ImageWidget (which contains an image), up to ComponentWidget, which contains a JComponent. This is really flexible for most cases; nevertheless you can subclass Widget if you want to do something special.

Which is precisely what I need. In spite of the fact that a Thumbnail is an image, ImageWidget is not ok for me, since rendering Thumbnails is somewhat complex because of their asynchronous behaviour. So I'm going with the creation of a custom Widget with specialized rendering capabilities:

public class ThumbnailWidget extends Widget
  {
    private static final ThumbnailRenderer thumbnailRenderer = new SimpleThumbnailRenderer();
    
    private final Thumbnail thumbnail;
    
    private final Node node;

    public ThumbnailWidget (final Scene scene, final Thumbnail thumbnail, final Node node)
      {
        super(scene);  
        this.thumbnail = thumbnail;
        this.node = node;
      }

    @Override
    protected void paintWidget()
      {
        final Graphics2D g = getGraphics();
        final AffineTransform transformSave = g.getTransform();
        final Rectangle bounds = getClientArea();
        thumbnailRenderer.setThumbnail(thumbnail);
        g.translate(bounds.x, bounds.y);
        thumbnailRenderer.setBounds(bounds);
        final double zoomFactor = getScene().getZoomFactor();
        g.scale(1 / zoomFactor, 1 / zoomFactor);
        thumbnailRenderer.paint(g);
        g.setTransform(transformSave);        
      }
  }

Keep in mind that you aren't forced to use Nodes with the Visual Library: but the next code samples will demonstrate why it's a good thing to have them behind the scenes.

Rendering happens in the paintWidget() method, where you can retrieve a Graphics2D object and paint all the stuff. Since a Scene can be zoomed in and out, it's important that you deal properly with the current scale.

The second thing to implement is the capability of automatically update the Widget when the Thumbnail state changes. Since this capability is pretty important throughout blueMarine, I've already told you that Nodes coming out from a ThumbnailTracker have automatic update capabilities. So what I need now is just a NodeListener:

    private final NodeListener iconChangeListener = new NodeAdapter() 
      {
        @Override
        public void propertyChange (final PropertyChangeEvent event) 
          {
            if (Node.PROP_ICON.equals(event.getPropertyName()))
              {
                repaint(); 

                //
                // The Nodes API can fire events outside of the AWT Thread
                //
                if (SwingUtilities.isEventDispatchThread()) 
                  {
                    repaint();

                    getScene().validate();
                  }
                else
                  {
                    SwingUtilities.invokeLater(new Runnable() 
                  {
                    public void run() 
                      {
                        repaint();
                        getScene().validate();
                      }
                  });
                 }
              }
          }
      };

// in the constructor of ThumbnailWidget:
    node.addNodeListener(iconChangeListener); 

repaint(), as the similar method in JComponent, just causes the current Widget to be repainted (another method, revalidate(), should be called if you have changed the size of the Widget, which is something I'm not doing here).

Now I would like to have a popup menu working on my ThumbnailWidget. By default, they don't have one, but you can specify a PopupMenuProvider for this purpose:

    private final PopupMenuProvider popupMenuProvider = new PopupMenuProvider() 
      {
        public JPopupMenu getPopupMenu (final Widget widget, final Point location) 
          {
            return node.getContextMenu();
          }
      }; 

// in the constructor of ThumbnailWidget:
        getActions().addAction(ActionFactory.createPopupMenuAction(popupMenuProvider));

As you can see, the code retrieves the context menu of the Node. This is pretty neat, since I'm just transparently getting the actions that have been configured in the NetBeans platform; in other words, I'm being consistent with the fact that whenever I have something that represents a photo, I always get the same functionalities.

Furthermore, I can take advantage of the Nodes API. For instance, I'd like to retouch a bit the popup menu. In facts, it contains an "Add to light table" action that I've defined elsewhere - trust me, I'm not giving the code for this since it's not part of the Visual Library stuff. But there's no meaning in executing that action on a Widget that is already in the Light Table. On the contrary, a "Remove from light table" action would make sense. I can easily exchange the "add to..." with the "remove from..." actions by using a FilterNode:

    private class ActionFilterNode extends FilterNode
      {
        public ActionFilterNode (final Node node)
          {
            super(node);    
          }
        
        @Override
        public final Action[] getActions (final boolean context)
          {
            final List<Action> actions = new ArrayList<Action>(Arrays.asList(super.getActions(context)));
            
            for (int i = 0; i < actions.size(); i++)
              {
                final Action action = actions.get(i);
                
                if ((action != null) && AddToLightTableAction.class.equals(action.getClass()))
                  {
                    actions.set(i, SystemAction.get(RemoveFromLightTableAction.class));
                    break;
                  }
              }
                
            return actions.toArray(new Action[0]);
          }
      }

// in the constructor of ThumbnailWidget:
    this.node = new ActionFilterNode(node);

Now I'd like to have my ThumbnailWidget brought to front when a simple mouse click is performed on it. On this purpose, I can create a custom WidgetAction as follows:

    private static final WidgetAction.Adapter bringToFrontAction = new WidgetAction.Adapter()
      {
        @Override
        public State mouseClicked (final Widget widget, final WidgetMouseEvent event) 
          {
            if (event.getButton() == MouseEvent.BUTTON1) 
              {
                widget.bringToFront();
                return State.CONSUMED;
              } 

            return State.REJECTED;
          }
      };

// in the constructor of ThumbnailWidget:
    getActions().addAction(bringToFrontAction);

I think that the above code is self-explaining.

Now I'd like that my ThumbnailWidget can be moved by just clicking and dragging the mouse. This is really easy: I just need to add this in the constructor:

        getActions().addAction(ActionFactory.createMoveAction());

And what about resizing? The Visual Library provides out-of-the box support for resizing a Widget. You just need to add the related action in the Widget constructor:

        getActions().addAction( ActionFactory.createResizeAction());

At this point, by pointing the mouse on the Widget border and dragging, it gets resized. Easy! But it's not good for me: my ThumbnailWidget represents a photo and I don't want arbitrary resizing that changes the aspect ratio of the image. Fortunately I can put a constraint with the following code:

        final WidgetAction resizeAction = ActionFactory.createResizeAction(resizeStrategy, 
                ActionFactory.createDefaultResizeProvider());
        getActions().addAction(resizeAction); // add this BEFORE createMoveAction()

A ResizeStrategy lets me override the actual size the will be applied to the Widget:

    private final ResizeStrategy resizeStrategy = new ResizeStrategy() 
      {
        public Rectangle boundsSuggested (final Widget widget, 
                                          final Rectangle originalBounds, 
                                          final Rectangle suggestedBounds,
                                          final ResizeProvider.ControlPoint controlPoint)
          {
            final Rectangle result = new Rectangle(suggestedBounds);
            final double deltaW = Math.abs(suggestedBounds.getWidth() - originalBounds.getWidth());
            final double deltaH = Math.abs(suggestedBounds.getHeight() - originalBounds.getHeight());
            final Insets insets = getBorder().getInsets();
            final int mw = insets.left + insets.right;
            final int mh = insets.bottom + insets.top;
 
            if (deltaW >= deltaH) // moving mostly in horizontal
              {
                result.height = mh + Math.round((result.width - mw) * aspectRatio);
              }
            else // moving mostly in vertical
              {
                result.width = mw + Math.round((result.height - mh) / aspectRatio);
              }
            
            return result;
          }
      };

The code looks a bit tricky, but the point is that I just need to return the new size that I want to apply. I'm basically enforcing the aspect ratio, just caring if the user is moving the mouse mostly horizontally (in this case I'd leave the new width unchanged and compute the height accordingly) or vertically (in this case I'd do the opposite). Also I'm taking the Insets in the computation, since they are usually non zero due to the capability of setting a Border to the Widget.

Yes, the Border. I actually want to change the border of my ThumbnailWidget dynamically:

• by drawing a solid, white border around the selected widget;
• by drawing a "resize border", with the classic eight control point where you should drag the mouse, when the mouse is hovering on the Widget.

This is also easy: I just need to override the notifyStateChanged() method, that is called whenever the Widget changes state:

    @Override
    protected void notifyStateChanged (final ObjectState oldState, final ObjectState newState) 
      {
        super.notifyStateChanged(oldState, newState);
        setBorder(newState.isSelected() ? (newState.isHovered() ? RESIZE_SELECTED_BORDER : SELECTED_BORDER) :
                                          (newState.isHovered() ? RESIZE_BORDER : EMPTY_BORDER));
      }

Just keep in mind that Borders for Widgets aren't the same classes as for the plain Swing component, and they have a specific BorderFactory. The details for creating a border are quite boring and I'll leave you to inspect the source code for them.

That's enough for this post. I need to run tests on the new component, but I've seen only minor glitches so far and I'm pretty pleased with the small amount of code that the Visual Library required to implement the features I had in mind. The most important point that should be addressed is the persistence of the Light View, that is a way to "remember" the items and their positions. It should be not hard since the Scene exposes methods for enumerating the contained Widgets, but I'll deal with this later - also because this first experience with the Visual Library made me thinking of some other cool thing that I could do with it... :-)

And remember that I just scratched the surface of the Visual Library: even if you need more complex renderings, including graphs with nodes, arcs, floating connections, etc... , the Visual Library has probably what you need.

Technorati Tags: NetBeans, Visual Library, blueMarine