Rory Winston's Blog

Happy New Year

Well, it's that time of year again. It's hard to believe that 2004 went so fast. This year has been busy, and downright hectic at times, so it's good to be able to sit back for a couple of weeks, take a breather, and make some plans for the next year.

The Java-related highlight of the year for me was making a trip to SF to this year's JavaOne. The best part about JavaOne was getting the chance to meet so many truly great people while attending. It's a funny thing about conferences - a couple of weeks after the conference is over, you can't remember what sessions you attended, but you always remember the parties (most of them ;))

I normally don't like to make predictions, but here's a few bets that I reckon are pretty safe for 2005:

Contrary to what Don Box predicts, I think that Firefox will continue to grow in 2005 and will not be usurped by a "new kid on the block" browser from a third party (i.e. Google). Browsers are tough to get right, and the market can be fickle. Firefox is, after all, only now benefiting from an investment put in a few years back (remember Phoenix)? The main challenge for Firefox will be to maintain the momentum that they have managed to achieve over the last year.
The EU legislation on software patents has been the subject of a bitterly-fought battle during 2004, and this will continue well into 2005. Many open-source (and commercial) developers are nervous about the impending legislation, and their nervousness has been made all the more acute by the seemingly underhanded way in which the legislation has been repeatedly pushed forward, despite heavy opposition. I'll be optimistic on this issue, and predict that the legislation, even if it is passed, may have much of its bite removed after a drawn-out EU parliament battle. The resulting legislation will be an expensive and ineffective attempt to placate both sides of the debate, whilst not really benefiting either. Much like a lot of EU legislation, really.
I would be very surprised if 2005 is not the year that SCO finally disappears beneath the waves. Like a lot of people, I have been following this madcap adventure (you couldn't make it up) since it began, and now it looks like it might be finally running out of steam. With alarmingly few allies in their corner, and investor confidence long since evaporated, they have become a running joke. Onegood thing to come out of this has been the Groklaw site, run by Pamela Jones, which proved conclusively that the open source model can work for more than just software. Maybe when this is all over, she will write a book about it. I reckon she will have no shortage of offers.

In 2005, I'm looking forward to more developments on the rich client side (a lot of competing technologies sprung up in 2004), the release of Geronimo, more exposure for JDK 5.0, and finally taking a look at EJB 3.

Happy new year.

Why I love log4j

I was wading through fixing a bug in an open-source package the other day, when I started thinking about the true value of consistent and detailed logging. Sounds like a no-brainer, I know. Well, check out the vast amount of open-source (and commercial, even if most of them won't admit it ;-)) code out there with little or no logging. Now, I'm frequently guily of this myself - it takes some discipline to stick with a consistent logging strategy, just as it does for a consistent test-driven strategy. But once it's done, it's done. The benefits are there for all for the duration of the life of the codebase, and it's much more difficult to retro-fit comprehensive logging on top of an existing codebase rather than develop with it from the outset (ditto for unit tests). Akin to this is the effort required to properly classify each message with the appropriate level, instead of blindly labelling every log statement as DEBUG or INFO. It's also easy to take for granted what a well-crafted logging framework can do - I never knew what say, a log4j Nested Diagnostic Context was before reading this entry in Gavin King's blog.

Which brings me neatly on to my next subject - good old log4j. It's easy to take it for granted, given what it does, it does so well, it tends to be ubiquitous, and logging's not really a very exciting subject, is it? (Not even when used in "Hello World" AOP examples ;-)). Nonetheless, I decided to take a look at what was going on with log4j development, and was mildly surprised to find it in a very healthy and active state. I hadn't realized that there was life after log4j 1.2.8, which has been out for a long time now, but the log4j committers have been very busy, and there are a whole host of new things coming in log4j 1.3, such as a new GUI (Chainsaw v2), enhancements to performance, configuration, i18n, and much more. There's also a little-publicized "Micro" version of log4j, called log4jMini, designed for J2ME-style applications.

It's the flexibility of log4j that I really like. Its functionality does make the java.util logging package look like the pale and watery imitation that it really is. We have a requirement on our current project to collect certain log categories and send them to a central syslog analyzer, in syslog format. With log4j, no problem - we can filter out the categories of interest and direct them to the syslog Appender, whilst sending them in parallel to the normal log files. A custom log level (e.g. "audit") and the JDBC Appender gives you a quick and easy auditing "framework".

And it's not just Java, either. A couple of months ago, I wrote my first piece of Perl code in about year. I was creating a Perl client for a secure Java-based SSO module. I decied to give the log4j-alike Log4Perl a try. This, as the name might suggest, is a pretty much a functionally equivalent implementation of trusty log4j. A typical Log4Perl configuration file might look something like this:

log4perl.rootLogger=DEBUG, A1 log4perl.appender.A1=Log::Log4perl::Appender::Screen log4perl.appender.A1.layout=PatternLayout log4perl.appender.A1.layout.ConversionPattern=%d %-5p %c - %m%n log4perl.logger.com.foo=WARN

The interesting thing about log4perl is that if you pass it a log4j config file, it will attempt to internally convert it to a log4perl style config automatically. A typical log4Perl usage looks identical to the Java version:

my $logger = Log::Log4perl->get_logger("SSO"); $logger->debug("Using cipher: $cipher");

Nice and familiar.

I've also found the Log4E Eclipse plugin to be very handy. It removes some of the tedium re: inserting and initializing Loggers, and it can even automatically log entire classes and methods (although YMMV here - it can be a tiny bit flaky when automatically adding and removing method-level entry and exit points). It does also work with java.util.logging, if for some reason you want to work with that.

Anyone else found any useful log4j scenarios? Written any unusual Appenders, or found a use for some of the more advanced features (like NDCs)? Leave a comment if you have.

JavaOne

So I've just got back from my first JavaOne, and I'm sorry I never attended previously. There were some great tech. sessions and BOFs (and a couple of dire ones - specifically a JSF tech session that degenerated into a marketing session ), but best of all, I got to meet a huge amount of people, and finally put faces to a lot of familiar names. Special thanks to the Geronimo, Groovy, Tangosol, Atlassian, Solarmetric and JDOGenie guys for some long nights of entertaining drinking at the local spots.

Some of the talks that I enjoyed most were the sessions on JDK 5 concurrency and memory model changes., as presented by Bill Pugh, Brian Goetz, and James Holmes. Also interesting were the AOP-related talks. There was an interesting session full of examples given by the AspectWerkz guys, and Ramnivas Laddad gave an talk on AOP and metadata. Some examples here would have been good, but I guess that AspectJ's annotation support is still in the works, so real-life examples would have been impossible. The annotation support and syntax in AspectWerkz looks great though, simple and intuitive. I got a demo of the AJDT's (AspectJ's Eclipse plugin environment) crosscutting visualizer from one of IBM's AspectJ guys, and that looks excellent. It's nice to be able to see just where your aspect is intersecting with your code. I also confirmed that the way I'm currently using AOP in my current project is a lazy and terrible hack, and I'm a very bad man :-). The AOP panel was great, if only to affirm that AOP in general can be a very useful and powerful tool, but is also very easy to misuse. The last thing you want to end up with is a bunch of long unreadable, difficult-to-maintain code being "patched" at runtime by long, unreadable, difficult-to-maintain aspects. JSF was a hot topic this year, and so was rich client-related stuff in general. We had some discussions on the SWT RCP stuff, plus a look at the Java Desktop Network Components project, which reminded me of Macromedia's Flex, and what they are doing with MXML (though the JDNC project is now open-source). JDNC looks interesting, it will be good to see what they come up with in the future. You can take a look at JDNC at https://jdnc.dev.java.net/.

I also found my way to the JCP awards, the words "free beer" ringing in my ears. James Strachan picked up an award for Groovy. His accepting words were "let's have more open source JSRs". Hear hear.

Speaking of open source, the debate on open sourcing Java was interesting, but ultimately kind of pointless. I can't really see one compelling reason right now to open-source Java. As James Gosling said during the debate, the bugs are all there for everyone to see, the source code for J2SE is downloadable from the website, the JCP process is continually evolving - if something's broken, I can't see it. I do think, however, that open-sourcing the TCKs might be a good idea - I don't see how this would be a bad thing for Sun at all.

Oh, and as a side note, this trip to SF was the first time I ever used Mac OSX. I think I'm converted - what an experience. I can't get over how much prettier Safari is on the Mac than its sisters (Mozilla and Firefox) are on Windows. I'm thinking of getting a PC for some (very) amateur music making, and was looking at getting a Wintel box, but maybe, just maybe....

Struts 1.2, Hypersonic and Hibernate

This week I've been playing with a couple of things that I haven't used before - Struts 1.2-dev for one, and Hypersonic, a 100% Java RDBMS that supports transactions, RI, and stored procedures. I have been pleasantly surprised with both.

I've always thought that Struts action mappings can be a bit cumbersome to write when there are lots of them. XDoclet can help here, but there are some new Action mapping techniques on the way in Struts 1.2. I like the new wildcard Action mapping features that Struts 1.2 offers: for instance, you can define an action like the following:

and the framework will automatically substitute, and look for, the corresponding match parameter. So the example above could generate an action mapping like:

There is also a nice feature, called mapping dispatch Actions. Basically this allows you to "overload" Action classes, and create an Action mapping to any method within that class that matches the Action::execute signature. As an example, you could write the following Action mappings:

Which Struts can automatically map to the following methods within the SubscriptionAction class:


 public ActionForward create(ActionMapping mapping, 
                             ActionForm form, 
                             HttpServletRequest request, 
                             HttpServletResponse response) throws Exception


 public ActionForward edit(ActionMapping mapping, 
                             ActionForm form, 
                             HttpServletRequest request, 
                             HttpServletResponse response) throws Exception

Using Hypersonic for development is excellent for proofing persistence work. Hypersonic is deceptively powerful for such a lightweight database. Just start up Hypersonic with runServer.bat and fire up the GUI manager with runManager.bat. The GUI manager allows you to run queries within the manager window and do some database maintenance tasks - very handy for a quick visualization of how your persistence code has just affected the DB contents. See a screenshot below:

All of this can be tied together nicely with Hibernate. Struts has a pretty simple plugin architecture and lifecycle. I used an existing implementation of a Hibernate plugin for Struts to initialize and destroy the persistence layer for the web app on startup and close.

These tools all work together quite nicely. I'm looking forward to Struts 1.2 being finally released. I' m looking forward to playing with JSF at some stage, but in the meantime, the momentum behind Struts doesn't seem to have slowed any at all.

CVS, BugZilla, AntHill, and Clover

This isn't a blog about Java-based technologies per se, but it's relevant to any Java programmers working on Open Source platforms. Recently, I have spent a few days working on migrating our existing CVS repository onto some new hardware. This time, I also set up ViewCVS and BugZilla, and took some steps to integrate them more tightly. I worked from a great overview by Steve McIntyre, which is located here.. Integrating CVS and BugZilla is a bit of a tedious process (currently, it involves patching and recompiling CVS, patching BugZIlla, setting up a user for BugZilla mail, etc.), but the results are worth it. I believe some patches are in the works for BugZilla to make this process easier, but for now, it takes a bit of time.

Once you have gone through the process, you can insert bug references into CVS commits. The corresponding bug report is updated with the details of the CVS commit, and bug update notifications are mailed out to the interested parties. What's neat is that a link is included to the modified file(s) in CVS , and also a link to a ViewCVS diff - very useful. A sample bug update is shown below.

A nightly cron job keeps the CVS modules file updated with the current contents of the repository.

The next step was to install a continuous integration / build manager. There are a couple of Open Source choices - I went for AntHill OS, as it is quick and easy to get set up. Using a continuous build manager is a good step - as well as build reports, you can generate docs, unit test reports,and code coverage reports. It allows you to monitor a project as a whole as it is being developed. Having a centralised build location allows you to to automatically notify an entire team about a build failure.

Code Coverage setup was the next step. I weighed up a few options, and went with Clover, which generates excellent reports, and is very configurable. Clover isn't free, but at the end of the 30 day eval, I'm going to buy it - it's definitely worth it. Just add it into your build files, and you're away. Which brings us to the next step.

It's not widely known how configurable ViewCVS is. It is entirely template-based (like BugZilla), so you can modify it as much as you want. I previously used the templates to add some project size information. This time, I wanted to be able to view the coverage reports for an entire project easily, so I made a couple of modifications to the ViewCVS templates, and now for all top-level projects, ViewCVS generates a link to their Clover reports. Clicking on the top-level Clover link will open the corresponding reports.This looks like:

These could be linked up even further - for instance, when a new project is created in CVS, a new product could be added to BugZilla, and you could add a further link on the ViewCVS top-level page to find all the open bugs for that project instantly - or maybe something along the lines of a JIRA-style roadmap (example) - a very cool feature.

There are a lot more features that could be added, but this is a good start. Spending a bit of time at the outset can help you work more productively in a team environment.

JDO 2.0 vs. EJB 3.0?

There is a lively discussion going on at TheServerSide at the moment,concerning the recent vetoing of JSR 243 by three large vendors - IBM, BEA, and Oracle. The reasons cited by the three vendors basically boil down to the seeming overlap between JDO 2.0 and the upcoming JSR 220. Of course, I suppose it's not surprising that three app server vendors with a heavy investment in EJBs would be disturbed by the very idea of something like JDO becoming a standard.

BEA, in particular, had a questionable response:

"another release of JDO, whose market acceptance is essentially constrained to use with object databases".

I think current JDO vendors would disagree. Certainly even a cursory look at the JDO 2.0 spec will refute that.

Until we get our hands on the review copy of JSR 220, we have no definitive frame of reference to determine exactly what the overlaps are.

At the end of the day, it will most likely be the community that will swing this one either way. If people want this kind of functionality and make that known, the vendors will capitulate. What is basically comes down to is: would you like to be able to use the functionality of Hibernate and similar frameworks in a standard way? And would you wish to participate in a process that defines this for future J2SE inclusion? If so (or even if not), then pop over to TheServerSide and contribute to the debate.

O/R Mapping and Performance

I am a huge fan of O/R mappers like Hibernate and JDO. They insulate developers from the database mechanics, speeding up development and boosting productivity. They also add a layer of insulation above the database itself, which aids portability. However, insulating developers from the database layer completely is not always a good thing. Developers still need to be aware of the performance impact of the code they are writing at that moment. There are a number of issues that can arise. Here are a couple that I have noticed on a recent project:

Improperly configured or non-existant lazy initialization. An object with multiple dependencies and relationships with other objects may trigger a raft of SELECTs when instantiated. These other objects, in turn, may trigger more statements to initialize themsleves, and so on. Objects within a large inheritance hierarchy are also vulnerable,as are objects that have a one-to-many (e.g. a parent-child) relationship, where the cardinality on the 'many' side is huge. To counter this, lazy initialization can help. Most modern ORM frameworks will use dynamic proxying or bytecode enhancement to hand back proxies to objects that may not be fully initialized yet. As the object is used, further parts of the object graph may be loaded into memory transparently. This saves loading many child objects at the beginning, only for them not to be referenced at all. Even better, the lazy initialization should be configurable per persistent entity, so each persistent class can have its own lazy loading policy. One size rarely fits all in such situations.
Inability to use database-specific optimizations. I know one of the major pluses of O/R mappers is that they obviate the need to write straight SQL, however, sometimes, writing some straight SQL can be the best answer to alleviate performance issues. Every database vendor has product-specific optimizations, and accessing these can sometimes make a huge difference. Purists may be unhappy with it, but in the real world, being able to access the underlying query code is a huge plus. I'm glad that JDO 2.0 is taking this factor into consideration.
Cache Configuration. A good caching policy is vital. If there are a lot of objects that are relationship-heavy, and also, say a security framework is involved, which loads associated security profiles for each object instantiated from the database, the number of database round-trips will be huge. Tailoring a cache layer to absorb a lot of the most frequently-fetched objects will save a lot of time. Preferably the ORM framework will give you the flexibility to plug in the cache framework of your choice. e.g. Hibernate even allows you to plug in a cluster-enabled cache (OSCache has some clustering features). The ability to mark a cache as read-only and use it as a store for immutable objects will enable further optimizations.