Per-object workqueues - is this a thing anybody needs?
A couple of years ago, at OOPSLA '06, I think, I had a lot of fun hanging out with Jarda Tulach and Rich Unger and writing a generic library for enqueueing a batch of jobs that run against an object on a background thread.
The fun part was really getting to dig into the java.util.concurrent classes, to do it such that enqueueing work is always non-blocking and transparently handles scheduling of work on a background thread and delivering the results.
The basic idea is simple:
- In an IDE, the user is editing a file. A lot of components may be interested in that file. The Navigator shows a structural outline of the file, and should re-parse the file. A task list may want to parse it and show TODO items and errors. Error markers and hints should be reprocessed.
- When the user saves, or stop making changes - on a triggering event - all of the things that are interested in the file's contents should have a chance to get updated. But, if, say, the file is an XML document, you don't want every interested party to do a separate SAX parse of the document, or create its own DOM tree - that's not good for performance.
I didn't want it to be specific to files, so it was designed with generics - it's a generic library for enqueueing jobs to be run against an object of some type. It keeps a separate bucket of jobs for each object, and runs them all on a delay after the triggering event.
The specific use of it in NetBeans is in the DocBook XML module, which we were using at the time to write Rich Client Programming. Basically, I wanted a way where all of the parts of the IDE that were interested in the file the user was editing could pass in a SAX ContentHandler; the library would then run one parse of the document, and all of the content handlers would be invoked in that one parse.
public interface QueueWorkProcessor <Target, WorkType> {
public void process(Target key, Drainable <WorkType> work) throws Exception;
public boolean handleException(Exception e, Target key,
Drainable <WorkType> work);
}and then just create a
Dispatcher for it (Drainable is just a collection of jobs which can be emptied by type - so different things can handle different kinds of jobs - for example, in the DocBook module, there are both jobs that use ContentHandlers and jobs that use java.util.regex.Patterns - different code grabs the jobs of each type and runs them when the work is ready to be done). When you call Dispatcher.put (Target target, WorkType worktype), it gets enqueued - and put() never blocks and the work always gets completed (at least, so say my unit tests).
The question is, is this too esoteric and weird a thing to have its own project (you can already get it from NetBeans sources, after all), or is there someone out there who actually needs such a beast?
And as usual, there is the question what do you call a thing that does this? I've been calling it "workqueues" (you can find it in NetBeans sources in contrib/api.workqueues), but suggestions are welcome.
- Login or register to post comments
- Printer-friendly version
- timboudreau's blog
- 1159 reads






Comments
by pblemel - 2009-05-06 18:55
Yes it's useful, and scalable. Consider extending the interface to specify OutputType produced by the worker. Add a query to the registrar, and you can automate entire work flows. I coded up mine using a finite state machine. I didn't know that there was already a module for the worker part, or I would have used it instead of growing my own.by n_varun - 2009-05-05 22:46
Glad to see you are blogging again, as its been a pleasure reading them :) Nice post..