Skip to main content

Another important milestone: ForceTen is completely mavenized (or why I moved to Maven)

Posted by fabriziogiudici on October 6, 2009 at 9:45 PM PDT

Another important milestone in the process of switching to Maven all of my projects - ForceTen is my first NetBeans Platform application that has been completely mavenized (other mavenized blueMarine modules are just libraries of components, and there's no mavenized blueMarine executable yet).

ForceTen is a project containing the geographic components of blueMarine (maps, geotagging, etc). So far it hasn't been much more than a set of components, but it can be run on its own and features the capability of exploring a catalog of geographical items that can be rendered with many different 2D and 3D geo viewers.

Tonight I was going to apply some refactorings to the code in order to blog about some ForceTen - but the NetBeans IDE editor started mercilessly breaking preventing me from doing anything - I don't know what it is and it's too late for searching a fix. So, it's probably the good moment to blog about why I've converted to Maven. There are very strong reasons for this choice, even considering that tonight issue is only the n-th in a set of problems that I had since when I started in July - it's really frustrating some times, but I'm telling you why it's worth while.

My previous, Ant-based build environment was described in a post from November 2007, "Beyond suite chaining", and illustrated a technique to split a large project in sub-projects, in order to satisfy the divide-et-impera strategy (in Fall 2007 blueMarine was using the "suite chaining" technique and was split in three sub-projects, a technique adopted a few months earlier when I realized the project had grown too much).

In that post I introduced a set of in-house developed Ant tasks able to get a set of .nbm files and expand a platform out of them; in that way, every Platform project was compiled against its own custom platform (actually, ForceTen was created at that time, as the spin off of the geographic features, as well as OpenBlueSky, a library of reusable Platform components, some of which could be even pushed further back to PlatformX), making each project a completely independent compilation unit. Importing a module from a different sub-project was the matter of copying the .nbm files in the proper place (another Ant task was developed to make it easier, working in batches).

This approach worked well for 1.5 years - but in the latest Spring, two new projects were introduced (blueOcean and blueBill-server, two server-side projects that share the media management capabilities of blueMarine); in addition a non public, commercial project had been refactored to use the blueMarine components (it's important since it's the primary funding source for my FLOSS stuff). This explosion showed that I had a problem with the management of artifacts, specifically in the process of promoting a build of a module to use it in another project. In fact, my home-made Ant tasks worked fine in a scenario in which any new version of a module had to be immediately imported to upstream projects; something that was no longer necessarily true with the growth of their number. For instance, to become a valuable independent application, ForceTen would need some modules that have been developed in blueMarine and that should be "pushed down" to OpenBlueSky.

I needed to introduce a facility for creating a versioned repository of artifacts - but that's precisely what Maven does. I did evaluate Maven in the past years, every time concluding that it was too buggy, unmanageable and complex for my needs - but in July, the increased complexity of my projects balanced the complexity of Maven, and I made my choice. I can't afford to evolve and maintain another dependency management system (*), and people eventually wanting to cooperate to the projects would probably be put off by it.

There's another reason. After 4-5 years of serious work in the open source world, only a small fraction of my code is currently used as is by others (for the record, jrawio). For other projects, I eventually receive emails revealing that more people are using parts of code that I don't suspect, but feedbacks are rare and contributions near zero. Indeed, in the latest years I've written quite a number of Platform components that I think could be profitably used as they are by other people. With Maven and dependencies declared at build time it will be possible for people to use the single component that they need, automatically importing the required dependencies. After I fix the last problem with the IDE and I'm able to make a few clean ups, I should be able to demonstrate the point with some modules related to the Visual Library. I hope it's a matter of one or two weeks to demonstrate something.

As part of the work of making reuse as easy as possible, I'm disclosing another sub-project I'm working on. It's the NetBeans Platform Wrapper Repository (NB PoWeR) - it's a Maven repository of pre-made wrappers for some commonly used libraries. It's a needed thing - how can you talk about reuse if you can't reuse the most trivial thing, that is libraries? Creating a wrapper library is not always a trivial thing: while in the simplest case you just need to correctly declare the public packages (but, I say, why each one of us should redo the same thing every time?), some libraries are more complex. For instance, the JPA wrapper contains some enhancements that I made explicitly for working in a Platform environment (e.g. assembling a persistence.xml file from many fragment embedded in multiple modules); also I'm going to publish to NB PoWeR the artifacts from the NetBeans OpenGL Pack by Michael Bien - they solved a big problem of mine, that is to have all the different native libraries working in multiple operating systems and 32/64 bit architectures.

The project it's still experimental, as it's clearly explained in the home page, but I'd be happy if you look at it and provide feedback.

(*) Yes, I've evaluated Ivy too and it's a good tool. There are other details in my build process that are greatly simplified by Maven, and now I'm not going to blog about them - they are related to the aim of making everything OSGi compatible. In any case, if you don't like Maven and don't plan to use it, don't worry, it will be possible to reuse my components with Ant + Ivy if you prefer.

Related Topics >>


the app looks cool. exactly

the app looks cool. exactly what I was looking for. I've written an Geocache ( manager for my Garmin Colorado gadget. It sends the gpx files to and ffom the device. However lately I've hit the gpx file limit on the device and need a way to figure what caches to delete. The low hanging fruit like "delete the ones we've found" only gave a bit of breathing space. It looks like plugin/module for ForceTen is exactly what I need :) BTW: it took me 4 hours to clone your repo, quite a list of binary files in there...

Binary files: I know, this is

Binary files: I know, this is the problem with Mercurial, as there are all the .nbm files that made the custom platform and that with Ant were committed in the Subversion repo. With Subversion it was not a problem, since you checked out only the working space; but now you get all the history. Do you know whether it's possible to strip from a Mercurial repo the past version of a set of files? hg strip removes the future, I need something to drop the past.

BTW, ForceTen is also supposed to provide the desktop end for windRose ( Let's talk about it :-)

Stripping historical binaries

No, you cannot remove past versions of files from a Mercurial repository, since that would destroy history. In principle Mercurial could allow you to just not pull those old changesets, but such a feature is not yet implemented: You could use 'hg convert' to create a new repo from which the history of those big JARs is excluded. The two repos will be technically unrelated, so you can't pull or merge across them, and changeset IDs will be different. For the NB build, we use to avoid keeping binaries in the repo. This would be less of an issue if we were using Maven since the build process would not be expecting binaries in the source tree to begin with. (If you need to maintain a custom Maven repo somewhere with binaries not found on public servers, this barely needs any versioning - all files should be write-once - so anything from RCS on up would suffice to track changes.)

back to the future

I don't know how Mercurial handles it but if you modify something in the past in a GIT repo the whole index from this point till HEAD is rewritten which makes merging from/to other clones difficult. (just as a warning) I removed all nbms as i moved from svn to git in past... was a one line shell command. I am sure hg has something similar.

PS In any case, the repo is

PS In any case, the repo is 189 MB. 6 hours are too much, probably Kenai had got some problems? I usually clone it in a matter of minutes.