Andreas Schaefer's Blog

Open Source Archives

3 Years later my Bitching became Code: Guilder POC Release

For the impatient readers I just wanted to announce the release of the Guilder POC which can be found on its wiki: https://madplanet.com/trac/guilder which intends to be a replacement for Maven 2 taking the cool features of Maven 1 and 2 and also incorporating some of the cool Groovy stuff to give building projects its Groove back. For all others here is the a little bit longer story.

In December 2004 I first complained about Maven 2 and I never really started to embrace Maven 2 and still think that Maven 1 is the better solution. But now Maven 1 is dead and Maven 2 took over. Eventually I had to use it in my line of work and discovered even more problems. Because I don't think that Jelly is a good scripting language and never started to be used outside of Maven 1 in a bigger way I was looking for another scripting language that I could use as its replacement. Eventually someone pointed me to Groovy and the Maven 2 Groovy plugin. Even though I liked the idea of Groovy the Maven 2 Groovy plugin did not work for me. I did not want to have a plugin that execute Groovy code but that could be written in Groovy. I also did not succeed in mixing Ant statements with script code as I could do in Jelly which took most of the appeal of Groovy away and so a lot of time passed by.

Well, somehow I stumbled over Gant which is Ant in Groovy and I saw in an example what I was looking for. Using closures and a Builder class Groovy was able to provide code that would exactly do that. Here is a simple example:

ant.junit( printsummary: true, haltonfailure: true ) {
    if( classpathName != null ) {
        classpath( refid: classpathName )
    }
    ...
}

As you can see there is Ant's optional jUnit task intermingled with the check if the classpathName is not null and then it will be set a class path ref for the jUnit task. Beside the fact that I can mix and match Ant with Script Code this code also looks nicer because I don't have to deal with the closing tags and with other XML oddities. So nearly two months ago I started to think about a project that uses Groovy able to replace Maven 2 with scripted plugins. Now you might think that why replacing Maven 2 if I just could add the ability to create and deploy Groovy based plugins. The answer is that there is more that I don't like in Maven 2 than just the compiled Plugins and I hope that eventually all of these shortcomings are handled when this project is released. Here is a list of such features:

• Scripted Plugins based on Groovy (maybe with JIT compilation)
• Plugin Context that exposes the user interface but also the interactions with other Plugins
• POM is a Groovy Script as well (no more XML)
• POM Is completely flexible and its structure is defined by the used Plugins rather than the Project
• Transitive Dependency Management using Ivy
• Local POM and local Plugins (see maven.xml)
• Goals can be overwritten or another goal can be executed before or after another goal
• The project builds itself
• ...

Then it took me around 4 weeks to finally get a POC release worked out and finally last Tuesday I was able to announce the release of it on the LA-JUG meeting. After that I heard from many other build tools that are growing like wheat: Apache's Buildr, Gant (heard from it before), Gradle and Graven. After a short scan I don't think that Buildr is a good idea because the instructions are written in Ruby but that's maybe just me. Gant is good but its Ant in a Groovy coat and Graven is the same for Maven 2. Gradle seems to be the closest to what I want to do but I think that there are still major differences between the two when it comes to the idea of plugins and probably other areas.

If everything works out as expected I should be able to release the Alpha version in a month or two which should have a solid core and a few full blown plugins.

Have fun - Andy

A Short Review of Maven 2

Tuesday, 6th of December 2005 I had the chance to attend Carlos Sanchez's presentation about Maven 2 at the LA JUG in El Segundo. I already played around with Maven 2 and therefore had a list of questions I tried to get answered. Especially I wanted to see if my concerns I expressed earlier in a Java.net blog where valid or not. Today I came to the conclusion that Maven 2 has some good ideas but I think that the Maven team threw out the baby with the batch water in some of the new features/restrictions and so I still prefer Maven 1 over Maven 2.

These is a list of the problems I have with Maven 2:

1. Properties cannot be set outside the POM
2. Any dependencies must be provided by repository (local or remote) and cannot be redirected to a directory within the project.
3. For now a project cannot reuse Jelly code from a Maven 1 project and so must convert that code into a Java based plugin.
4. The build hierarchy cannot be extended by the project (no pre or post goals) and the order of plugins executed within one build step is undefined.
5. The current Maven plugins use Java classes and the plugins downloaded do not include the source. The source archives of these plugins then do not include a build environment just only the Java source code.

1. Properties cannot be set outside of the POM

Many open-source projects use other applications to build, deploy or execute their application. Most of the time the project needs a path to this application like for example the JBoss deploy directory or the installer root directory. On the other hand the project cannot know the path to these directories and therefore does not want to include these directories into their version control system(VCS)..The only solution is that every user does set the path in a separate properties file like the build/project.properties file in Maven 1.
For me this is a must have requirements and I do not see another way around it.

2. Dependencies must be in the Repository

Not every open-source project has their archives deployed in the Maven repository or is able to run their own repository. Also sometimes the Maven repository is not up to date or cannot provide archives for other reasons. In that case an open-source project needs to add dependencies in their VCS and add the dependencies without the repository. Currently in Maven 2 a project can only ask the users to copy the archives in question to their local repository in order to make the build work.
I could accept that if Maven 2 would be able to have a project internal repository that can be made available through the VCS.

3. No Jelly in Maven 2

In Maven 2 one cannot inline or execute with a plugin any Jelly code one might have from a Maven 1 project. Even though that I do not like Jelly as a scripting language it really makes sense in conjunction with Ant because their XML code can be easily intertwined making it easy and fast. Maven 2 allows user to inline Ant and so I was wondering why not also Jelly.
This is a must have requirement for me.

4. Build Hierarchy

In Maven 1 it is easy to hook into the build hierarchy (defined by the plugins), extend the build hierarchy or do overwrite it (somewhat like in AOP) with what I need. Maven 2 does not allow me to do that anymore and so I am stuck with the build hierarchy. For example if I would have a code generation tool's output is the others code generation tool's input then I have a problem because I cannot define an order in that step. Whatever build hierarchy Maven 2 has there will be projects that do not fit in. As much as I agree that many developers misuse Maven 1 there are still valid exceptions out there that do require a more flexible build hierarchy especially when Maven 1 had this flexibility.
A tool must make the life of a developer easier and that is just the opposite.

5. Source of a Plugin

Having the Jelly script with the Plugin in Maven 1 as the one single feature that sold me to it. It helped me tremendously to understand Maven, was a great source of how to do it and an easy way to debug it. Now with Maven 2 the plugins are written in Java and they do not contain the source and even the source archives does not contain a full build environment. I think that is quite a hassle and is not user friendly at all. I also do not understand why it was not possible to write a Jelly compiler if performance was an issue.
Java has proven that speed is a relative thing and so I do not see the need for Java plugins. I saw the problems of Java based plugins in the XDoclet project and would loved to see them using a scripting language instead. Now Maven 2 does the reverse with minimal gain but a lot of pain if things do not work as expected and that will happen quite often.

Conclusion

Beside the fact that Maven 2 made it to be released the other concerns are still valid leaving me no other choice than to keep using Maven 1 for now. In my opinion the team failed to meet their own goal of making Maven 2 simpler and easier to use than Maven 1.

Have fun – Andy

Open-Source vs. Big Bucks: II

I respectfully disagree with Jacob's view to the JBoss licensing issue. When I and even more true for Rickard started to contribute JBoss had a GPL license. Then Marc asked us if we agree to change it to LGPL but did not mention that he was going to register JBoss as a trademark. Therefore we were not aware of the issues that did arise now. The JBoss Inc. already used legal threads to stop competition with the Apache Geronimo project and so I think that is not a isolated incident.

I do no dispute the use of a trademark in a business but here an open-source project, a commercial entity and a trademark are combined in a such a way that the JBoss Inc. can profit from the open-source project without any competition. Matthias, from the JBoss Issue Blog, pointed out that Bea registered its name as trademark for the product and services but Weblogic is only a registered trademark for the product. This way I can advertise training for the Weblogic application server as long as I do not use the Bea trademark with it.

Even thought this issue that came up with JBoss it is a much bigger issue than that. It has implication on how open-source project conduct their business and that contributors have the right to know upfront on what their are getting into. This is also a question about ethics, control and protection of an open-source project. I always thought that the trademark of Linux and JBoss are there to protect the project from a sleazy opportunist that uses the trademark to exploit a successful open-source project. Now it seems that this one comes within its own ranks and that the idea of the protection was just a nice cover.

JBoss Inc. can make as much money of the project as they like and I am perfectly happy with that. On the other hand I know more about JBoss than most of the people with a JBoss certification or partnership with the JBoss Inc. so why shouldn't I be allowed to give training about JBoss?

Have fun - Andy

Open-Source vs. Big Bucks

What a disappointment! After feeling betrayed by the JBoss Inc. with the trademark lawsuits now also Linus Torwald seems to do the same thing with Linux. Yesterday I was attending a presentation about Java 6 (Mustang) and some of the discussions turned around Sun and open-source. Finally, I guess, I have to admit that maybe Sun is more trustworthy company with respect to open-source than some of the bigger open-source projects. I really feel that Sun is paying more attention to the open-source community lately than projects that started as open-source projects and then became a business.

I am especially disgusted by the comments of Sacha Labourey stating that Rickard is not a co-founder of JBoss. Today there would be no JBoss if Rickard Oeberg did not revamp early version of Marc's monolithic application server. He was the one that came up with the use of JMX as an IOC framework, the use of Interceptors to easily wrap J2EE services around an EJB call and the use of Dynamic Proxies to avoid the generation and distribution of stubs. Without Rickard JBoss would have ended up the same way Jonas did. Of course Rickard is not a co-founder of the JBoss Group (today called JBoss Inc.) but they were created way after JBoss became a serious J2EE-based application server.

Maybe there is a need to separate the open-source code from any trademark to prevent individual from taking advantage of code contributed by the community. I do not contemn anyone from making money of an open-source project but it is not acceptable in my opinion that this only applies to a few. I always thought that the JBoss trademark is there to avoid a big company from screwing the open-source project but now it seems that it backfired for the community and the JBoss Inc. can just cash in.

Have fun - Andy

JBoss: Big Buck vs. Open-Source

Rickard Oeberg's (co-founder of JBoss) blog points to the JBoss Issue Blog where he and Matthias Bohnen discusses issues related to JBoss. So far Marc Fleury and the rest of the JBoss Inc. could hide their business agenda behind the JBoss open-source project but finally they feel that they can afford to let the business take over because they are now so successful. On the other hand it finally vindicated the critics of the JBoss enterprise warning that in such a tight integration of open-source and business one day the idea of open-source in JBoss goes down the drain. Therefore I decided to join their open letter to JBoss and Marc not to abuse the JBoss trademark because many of use helped them to accomplish the current state. Without Rickard Marc Fleury would be still a pre-sales engineer at Sun because JBoss would not exist. If you think that JBoss should remain a healthy open-source project please join me by sign up to this open letter.

That said I still think that competition will eventually force JBoss to be more forthcoming and so I decide to start a thread evaluating Geronimo to see how Geronimo compares to JBoss. I would like to figure under what conditions Geronimo could replace JBoss and to point out their respective advantages over the other server.

Have a nice weekend - Andy

JBoss entering the Evil Empire

Today I just stumbled over the press release of JBoss announcing a partnership of some sort even though it is not clear to me what this means. This release does not surprise me from a business view but I am surprised from my past experiences working on JBoss. I can remember the first JBossOne in 2002 when one presentation showed the "screaming monkey" making fun of Steve Balmer. On the other hand I never thought that JBoss would become the wedge between Java and Sun. It seems that Marc Fleury became the "Drew Rosenhaus" of the Java open-source community using the movement to advance his agenda. I would not be surprised when one day JBoss would be bought by Microsoft. Luckily there is still Geronimo out there and I definitely have to have a look into the project now.

Have fun – Andy

Podcasting: The Hottest Thing on the Planet?

In case you listened to Steve Job's Keynote presentation at the WWDC you probably noticed that he spoke about potcasting and that it is a hot thing. Now when Apple is jumping on the podcast bandwagon and probably Microsoft is doing it quite soon then it must be a cool thing, don't you think.

Apple's plans are to include a podcasting aggregator into iTunes that is linked with their music store so that one can access the podcasts quite easily. Nevertheless if you need some extended features or having more control over what you get you may want to use a different aggregator to access and download podcasts. Most of them provide a way to put the podcasts into iTunes or other players anyhow. In happened that I am one of the developers on a Java based podcast aggregator called jPodder (please do not confuse it with iPodder which is Python based). We just recently released version 0.9 having very advanced features like ID3 tag rewriting on MP3 files which you may see as columns in iTunes or plugins to support many players instead of one like iTunes. I am considering writing a plugin that stores podcasts directly in an iPod rather than going through iTunes.

Now why would someone use a podcast aggregator if the stuff is already put into iTunes. A reason maybe the same as why someone is going through the hassle to use Firefox rather then Internet Explorer (on Windows). But there are others more subtle reasons to do so. iTunes is tightly integrated with their music store and therefore the features provided are there to either lure you into Apple's business model or facilitate that business. That said everything that is threatening their business is not going in there. In addition features that have nothing to do with their business are only being added to iTunes if they help them to increase their market share.

Imagine that you have a video player (or any type of portable mini computer) and you not only want to listen to podcasts but also see videocasts. These videos are going to be large and you are probably not going to wait for them to download until you can watch them. Nevertheless you could download them over night and watch them afterwards whenever you like. In addition a video player can play podcasts, blogs and maybe other stuff that comes along. This means a good news aggregator (one that can handle RSS feeds) would be able to deal with all sorts of content and you can configure that aggregator to download all the stuff you are interested into and the very next day you head out into the world with all the stuff on your player ready to enjoy.

Still in the near future Apple or Microsoft is providing features in their players to deal with additional content but Apple will keep iTunes simple meaning that you are limited in how to manage the media files and the download process. In jPodder we want to give the user the ability to configure it to their needs so that one can manage a big number of files with various sizes and content.

Have fun – Andy Schaefer

jPodder: your Pod's best Friend

I am delighted to announce that several months of hard work finally paid off and we could release jPodder 0.9 last Sunday. If you like podcasting or where wondering what it is or where not quite happy with your current Podcast receiver then you should read on.

Podcasting is same as Blogging except that it handles media files instead of text. It enables us to become our own Radio Station but also listen to anyone who has something to say, literally. Under the hood both uses a RSS (Really Simple Syndication) feed to publish their content and so any News Aggregator can read it and present it to us. Note that a web browser is not a News Aggregator because weblogs do provide the content as HTML rather than as a RSS feed document (XML) to a browser.

Now jPodder (do not confuse it with iPodder) is such a News Aggregator tailored towards audio podcasts. Because media files tend to be huge and some providers maybe slow such a specific News Aggregator does make sense because it also you to download media files in the background or over night so that you have a fully charged Pod in the morning to listen to throughout the day. jPodder was designed in such a way that you can manage a huge number of podcasts and even produce your very own podcast. Such features include (but is not limited to):

• download podcasts in the background
• able to run in the background
• allows you to rewrite the tags of audio files like Artist, Album, Genre etc either for all podcasts, for all podcasts of a particular feed or per podcast
• supports BitTorrent data transfer (peer-2-peer downloads)
• allows you to produce your very own podcast

You may wondering what rewrite of tags of audio files mean. An MP3 file (the one support by jPodder) contains tags (called ID3 tags) like Artist, Album or Genre describing the audio file in more details. A player like Apple's iTunes does use these tags when it lists the audio files. Now tag rewriting enables you to customize these tags to your needs so for example you can set the Genre of all podcasts to 'Podcast' so that when you list them by Genre you can easily separate them from the regular music files you have. Or you can set the current date in the Title so that you know how old a podcast is and therefor you can listen to them in order you received them.

If you are into listen to podcasts then I think it would be worth while to check out jPodder on www.jpodder.com. In case you are not happy with it we will refund you the full purchase price.

Have fun – Andy

Ever Heard of Ant2, XDoclet2 or Maven2

Lately I was made to look into Maven 2 and started wondering why so many complete rewrites of popular open-source projects never make it. In my view it comes down to the resistance of people to changes which has nothing to do with software development at all. In case you do not believe me try to explain why most PC users still are using Windows with all its problems instead of switching to Linux that became a real alternative to Windows. Now you may ask why did the people start using the first version of these projects if they do not like to change. My answer is that they did not until the pain became so strong that they were willing to give up their comfort zone and tried something new. From then on they waited again until the pain increased to a level they had to leave the comfort zone again. Of course, they could only leave when an alternative was available but then the acceptance would be overwhelming like with Maven.

The success of the first version of such projects is the problem of the successor. First they stand in the shadow of the original project and have problems to get attention from the developer and user community. In order to get the attention they need to have a lot of new features and improvements which, on the other hand, do backfire because we, the users, are used to the original version and tend to stay with what we know than chartering unknown territory. In turn that delays the development of the second version that now also has to keep up with the development of the original version. On the other hand they cannot completely change the project otherwise they would drive off many users of the original version jeopardizing their acceptance.

For example in Maven the most important feature to me are the plugins and that they are written in a scripting language (Jelly). This makes it quite easy to figure out what a plugin is doing, how I can use it and I am able to change it easily when necessary even if this is just debugging. Now in Maven 2 they drop the Jelly plugins replacing them with Java plugins that are faster but harder to read, debug or change. I would not care that much if I did not have seen this as the major problem of XDoclet. There they used an XML file to defined the XML deployment descriptors using special XML tags, an early version of Jelly, and their tag handlers are Java classes making it hard understand what is going on, harder to debug and very difficult to change. This is one of the reasons why the XDoclet 1.2 release was postponed over and over again.

Therefore I will not change to Maven 2 because Maven is still a very active project, it is easy and flexible and I still fight to replace many Ant projects with Maven.

In case a rewrite takes to long to be accomplished any other project solving the problems will take advantage of that and become a success early on like Maven as replacement of Ant. My conclusion is that it is maybe better to start all over again instead of trying to rewrite such a project. In order to be successful the project need to relieve users of their pain and therefore has to wait for a considerable amount of time until the initial euphoria is replaced with the agony of the daily grind. It is maybe better to focus on improving the original version until it becomes mainstream before starting with a new project.

Happy coding – Andy Schaefer

Maven: 'I love you' afterall

Correction:

At the bottom of my weblog I complained that I cannot redistribute JAXB with my project forcing others to download and install JWSP. Ryan Shoemaker from JavaSoft told me that this is not correct and an updated FAQ entry about this is made available. I am glad to hear this and hope that JAXB can be found on the Maven repository soon. I hope that this confusion helped to clarify this important issue and I want to thank Ryan for his valuable feedback.

The first time I saw Maven I did not like it at all. It seemed to be a huge, complex and unfinished project. But somehow I could convince myself to try it out in my very own JDoppio project. So I checked out the Maven website documentation and wrote the initial project descriptor. Now I just had to write the code, jUnit tests and was ready to run the project. So instead of writing an Ant script taking me roughly around one hour I wrote the project descriptor in ten minutes without loosing any features.

Then I decided to split up the project into multiple sub projects and found some existing projects using Maven's multi-project tool like Turbine. This took a little bit longer to understand the multi-project but to write the sub projects was quick and easy as before. After that I needed to import sub projects into other sub projects which was easy using the local repository and the dependencies in the project descriptor. Using JAXB made it mandatory to add my own archives to the local repository. Finally using properties in the project descriptor allowed me to add certain archives to a class path or to copy them into the test environment to run the JDoppio server standalone. So every requirements by my project could be resolved by Maven very elegant without making it complex.

The biggest challenge to me was to find help on problems I wanted to solve with Maven. The first stop was their website but it only works for simple questions. But then I discovered that looking into Maven plug ins and their project descriptors and Jelly scripts provided me with a wealth of information and every time I look into a different plug in I find new ways to solve a problem. It also helps to look into other open-source projects using Maven and their project and extension (maven.xml) descriptors to see how they solved a problem.

The only problem I found so far was some buggy plug ins which I either did not care about or got the newest version from CVS to solve an issue. Otherwise I am convinced that using Maven is a real time saver and if you flexible it can be used in most projects I can think off.

Have fun – Andy

P.S.: Can someone explain to me why Sun does not allow JAXB to be distributed freely and I have to bug all other developers interested in my project to download and install Java Web Services Developer Package to just get this archives? As if only web service developer would be the only ones interested in JAXB. Sometimes I am really wondering why Sun cannot stop making our lives miserable even thought they are using open-source software in their distribution as well. So Scott, please could you prevent JAXB from becoming the next Java on Linux issue where it took Sun years until they went along with the developer community.

A Generic Application Server

Since over a year I am thinking about the next generation Application Server that takes the route of JBoss as an flexible J2EE based server and just go through with it and make everything generic. In this server everything is deployable like Containers, Services (as of Transaction, Security, Persistence, Pooling etc) and Applications. The application server specification is replaced by a contract each deployment unit exposes with their requirements and features they offer. The server will then match the components together so that a fully deployed application is formed. If a component is missing a deployment unit will remain inactive until everything becomes available. Even a naming service is deployed and therefore one server can hosts multiple naming services at once.

Such a server allows an administrator to tailor the server to the hosted applications needs and nothing more. So if no Entity Beans are needed this Container must not be deployed. If an Entity Container is needed based on JDO either a particular JDO Container or a JDO Persistence Service is deployed. It would also allow a Container to expose the Home and Remote Interface in a JMX fashion where a global interface exposes the list of supported methods as well as their invocation. I guess, this would make Web services much simpler.

For more information about this ideas please feel free to visit my personal log.

This work is licensed under a Creative Commons License.