Vikram Goyal's Blog

Dependency hell

I started work on an existing project at my day job today and the first thing that I had to do was to download all the dependencies for the project. Here is a snapshot of the dependencies folder after I had checked it out of CVS:

Wow! That's 21 dependencies in that folder. Twenty-One! Actually, it should be more because Jakarta-Commons contains many separate libraries, which in this case is 14 more. So that's a total of 35 dependencies I have to contend with before I start on the project.

It is a shock to me (maybe not to the rest of you) because I am coming back to working mostly full time on a Java project after a year or so, in which time, I was working on a .NET project.

I will say this now, looking at that folder: even though Java is my preferred platform, I never encountered this kind of dependency nightmare working with .NET.

What's the maximum number of dependencies that you have encountered? Is it a problem in the Java language that requires these many dependencies? Because to be fair, these dependencies wouldn't be there if they were not required.

And yet, I am almost nostalgic for the simplicity of .NET. *shudder*

Comments

• the eclipse & jonas dependency graphs are pretty fun:

  http://wiki.debian.org/Java/DevJam/2005/Oldenburg

  cheers,
  dalibor topic
  

  Posted by: robilad on January 17, 2006 at 08:45 PM

• At least it means there are a lot of stuff you won't have to develop by yourself in your project :)

  Posted by: gfx on January 17, 2006 at 10:41 PM

• 21 is nothing! I just counted the library part of the project in front of me and end up with 153. (ok, our own stuff is included :) )

  Posted by: herkules on January 18, 2006 at 05:13 AM

• I can't see this as a language based problem. I don't now .NET very well but there is a whole lot of libraries in the .NET assembly isn't there? If that is correct, it just means that you are working on more standardized set of dependencies.
  With availability of more and more quality open source projects it becomes more tempting to use these efforts instead of crunching out design and code yourself to solve common problems. So basically I categorize this as a project/configuration management problem not tied to the language as such.

  Posted by: gretar on January 18, 2006 at 06:54 AM

• Apache commons is the worst offender. I'm glad to hear others find this inconvenient. In my experience commons is the worst of the worst when it comes to having unecessary dependencies...so much so I wonder why they release them as individual jars, they should just be globbed together in one large monster jar because you end up depending on all of them anyway. For instance, almost any commons package requires commons logging or log4j.

  We've known how to solve this problem, IoC, there's no reason why any jar file should require a specific logging toolkit. It should just say "tell me which logging provider to use and I'll use it." I'm also not sure a reuse library should emit logging. When calling a 3rd party method, it should tell the caller if it encountered errors and leave it up to the client code to decide if it wants to log or not.

  Posted by: tcowan on January 18, 2006 at 07:00 AM

• Why do you consider it a nightmare? Is 21 dependencies too many? If so, how many are OK? 2? 10? 17? 18.35326?

  It looks like most of those are various open-source libraries which provide you valuable functionality, which you didn't have to write yourself. I'm not sure I get what the problem is.

  Anyway, if you're particularly worried about keeping the jars around and what-not, just download the source for those libraries, and merge the code into your project. Of course now you'll have more problems receiving updates and bug-fixes, but at least you won't have a lot of dependent library jars floating around.
  

  Posted by: sprhodes on January 18, 2006 at 07:01 AM

• Are you making a fair comparision with .NET in this case? When I worked with .NET projects, there were only the CLR run-rime and Visual Studio. No build tools (ant), no code-generation tools (xdoclet) no test-driven frameworks (junit, dbunit), no loggin framework (log4j), no object/relational framework (hibernate), no caching providers (ehcache), no mvc framework (struts), no IoC frameworks (spring), no presentation framework (velocity) and the list goes on.

  Although .NET provided some equivalent infrastructure, like web services framework (axis in your case) for most of the dependencies you have we had to not have the features or implement them outselves in our application code. Did you have equivalent infra-structure and libraries in your .NET project?

  It may be unpleasant to manage all those dependencies, and I think the JCP should coe with some standard for this area, but it's much better (easier) than writting all the code yourself or coding without those features.

  Posted by: flozano on January 18, 2006 at 07:10 AM

• 150 for me and it is just third party jars.

  By itself, I dont see it as a problem. The part of Java "enterprise" development we know and love is this huge amount of open-source goodness.

  In many cases, dot Net often does have an advantage - there is only one way of doing something so you dont have to pick and choose all the time. For projects with hundreds of people it is a good thing.

  Posted by: mgarber on January 18, 2006 at 07:27 AM

• Certainly not a fair comparison. I think you should first get into the project instead of first complaining and see what you are gaining from the libraries. If they don't fit the task at hand or if they are over kill for the given project then bring a good argument with you to trim the project down a bit. I've seen projects use many un-needed things, but then again there are times when they needed to use more. It all depends on the project not on how many dependencies there are.

  Posted by: wadechandler on January 18, 2006 at 07:34 AM

• Maven 2.0 could help you here with managed dependencies. I had around 30 libraries in a project, some used directly, some were transitive dependencies (dependencies of dependencies). I started having problems when I had to support two different versions on the same library.

  I then decided to look around for a solution and I tried Maven 2.0 and I must say I am just happy now to let this marvelous tool handles all this stuff for me :)

  Posted by: madtree on January 18, 2006 at 07:56 AM

• Try to compile some kind of common software (Apache with a bunch of modules, Firefox, Gnome, KDE, qmail...) from scratch on a blank Linux box. You will never ever complain about 21 little JARs in your WEB-INF/lib ;)

  Posted by: scotty69 on January 18, 2006 at 08:08 AM

• the same motivation of one of my blog entries....

  Maven is the best choice in this case..

  Posted by: felipegaucho on January 18, 2006 at 09:03 AM

• I disagree. Maven is only a good choice if your project fits the maven model. In my opinion, maven has some severe limitations. The first of which is the lack of support for multiple artifacts (which people have been clamoring about since the first version of maven). The only way to do this in maven is to have multiple pom files which have different module names. This is impractical for a lot of projects which package up their stuff differently.

  Take a simple example of a project which produces a war file, and a jar client file for remoting. Typically, one might name them similarly: a.jar and a.war. And would want them distributed together in the repository: my_org/a/1.0/a.jar and my_org/a/1.0/a.war.
  There's no way to do this in maven that i've seen.

  Secondly, i have a huge problem with the war plugin itself: It assumes you want your .class files directory in WEB-INF/classes, which in my opinion, is not the correct way to deploy an application. IMHO, it's much better to package up into a jar, and put that into WEB-INF/lib.

  Finally, trying to do anything custom in Maven 2 seems to me to be very involved, when compared to something like ANT scripting.

  
  If you don't like the model maven requires, I suggest taking a look at ivy:
  http://jayasoft.org/ivy
  It is purely a dependency management tool, and does not govern the layout, etc... of your project. It has support for multiple artifacts, multiple configurations, transitive dependencies, maven repositories, etc... Effective use requires that you're willing to create an ANT script to call into the included ANT tasks provided with ivy. There are some nuances to using it, but overall, it's very effective.

  Again, it boils down to what your project is trying to generate.

  Maven is not "the best choice". It is A choice which may or may not effectively work for the project.
  

  Posted by: mattinger on January 18, 2006 at 09:39 AM

• I don't understand either. You never explain why this is a nightmare. Are you saying all functionality should be written from scratch if it's not in the JDK?

  Posted by: keithkml on January 18, 2006 at 09:48 AM

• 
  I don't see what the issue is. You are choosing to be dependent on these libraries and they are helping you make your life easier. If you like, don't include commons-lang.jar, only then you'll have to write your own StringUtils class etc etc. Also, there's no gun to your head saying to use Hibernate - you can safely remove that dependency but then you're down to writing JDBC code. So the dependencies are helping you, not causing an inconvenience.

  
  I also work with .NET and the same "problem" exists there. For example, I wanted to use IBatis.NET, MySQL, Log4NET etc, so I needed the appropriate DLL's for them to work. Yeah, I'll give you that it's easy to "Add Reference" in VS.NET than to configure a build.xml but thats an IDE issue.

  
  To sum it up, you choose the dependencies so you'll have to write and test less code.
  

  Posted by: zarar on January 18, 2006 at 10:04 AM

• To mattinger:
  It just makes sense to me to have one project -one main artifact relationship so I don't have to filter my java class when the jar is created. I mean what is the advantage of keeping your client classes and your web classes together. In maven 2, you would just create a parent project (a multi-module project) with two children (one for the jar and one for the web). This way the objects share the same groupId. Then, just have two projects and put a dependency from the web project to the client project. This way you are sure you don't end up with not-used classes in your archive. By the way, in Maven 2.0 a project can produce secondary artifacts. For instance, the ejb plugin generate both the ejb.jar and the ejb-client.jar if you ask it. Well depend of everyone need but I don't like to have to rewrite build scripts everytimes. And if you really need the power of Ant, you can always call a Ant task from a Maven build. I think it's the best of both world :)

  Posted by: madtree on January 18, 2006 at 10:56 AM

• we'll agree to disagree. I've tried the parent pom thing, and it was very difficult to get correct when working in a development environment. Maybe they've actually made it work since the initial 2.0 release (which in my experience, it did not work properly). Either way, how would you tell maven to publish both artifacts? That's not somethign i've ever seen available in maven. You have to override the publishing goals yourself. IMHO, maven should have the ability to work with multiple artifacts.

  In any case, the two tools have overlap. Both manage dependencies, but in different ways. Ivy is purely DM, and not a build tool, which is what Maven is. One aspect of maven is DM. Personally, i prefer Ivy, simply because I find ANT more flexible, even though you do have to rewrite targets. I always seem to find some module who wants something extremely custom done during their build, and to me it's easier to do with ant. Perhaps it's just a learning curve thing. I'm not saying maven isn't useful. I'm just trying to point out the options, and my observations of maven's shortcomings.
  

  Posted by: mattinger on January 18, 2006 at 01:20 PM

• I don't understand the problem either.I see 25 pieces of functionality you don't have to write. Given your project stores these in CVS, resolving the dependencies is just a matter of synchronising your workspace with the repository... so - well, where's the beef?

  Posted by: phuego on January 18, 2006 at 03:28 PM

• when i read your first sentence i wanted to be sarcastic and suggest you using dotnet. few lines later it all became clear.

  no, it's not complexity or dependency hell, it is FLEXIBILITY. that's the way we, java programers, prefer.

  Posted by: ndario on January 19, 2006 at 12:56 AM

• Agree with phuego. Poor flozno who has to do all that work as part of is .net project since there are no ready-made modules to download. The .net runtime may be a little richer than rt.jar, but not that much.

  I don't like the rant about maven being not suitable for a particular project. I thought so too in the beginning. Now I realize that following the maven way of doing things actually is to follow the best practice, or at least a better practice.

  It is only a little unfortunate that the setup is called project.xml and not module.xml, since it leads people to think that a software project can get along with only one maven module. It can't. You need at least one for the java code and one for each of the deployment methods. No real software project should consist of less than four maven modules.

  Posted by: eirikma on January 19, 2006 at 01:06 AM

• I recently did some work on a .NET project (for a change!). I can assure you that the situation is no better. There are just as many dependencies on build tools and external libraries, but the dependencies sections in the Visual Studio project files are always causing problems. It is quite a common occurrence for everything to stop for an hour or two because builds fail due to a clash of dll versions.
  Having preached the idea of software reuse for a very long time I am reluctant to point out some of the pitfalls, but there are some. There are many occasions when a few fairly simple components are needed and actually implementing them would take less time and cost less than obtaining a library, understanding the way it works then modifying code to work in the way the library requires. On top of this we have the point that for component libraries to be viable they usually bundle a whole load of stuff together when you only require a part of it.
  For example when I first started writing code for XML processing years ago I downloaded and used JAXB, JAXP and a whole load of libraries to help. Now I just work directly with the DOM and SAX parser bundled with Java and life is a whole lot simpler.

  Posted by: mullsoft on January 19, 2006 at 03:14 AM

• Hahaha! I see that there have been no posts for the past 7 months about this issue... And I resisted posting... but my gut would not let me!

  Hopefully, the poster *has* gone back to .NET! I say this because he's comparing apples and oranges... To compare .NET to a tool like Maven is asinine. I imagine somewhere else he's comparing JEE with internet explorer!

  Let's clarify:
  Java is on the same plane as C#
  .NET is on the same plane as JEE
  Maven is on a plane by itself... It is more than a build tool (like make)... it is a *project management* tool.

  Some of those libraries in the reported repository may have been used by the tool itself, and not the project in question. It would take further investigation to answer the question...

  Posted by: machershell on August 30, 2006 at 05:49 AM

• 盛大金属制造有限公司是以生产钢管、无缝钢管、无缝管以及销售为一体得大型文件柜更衣柜流通企业。我们提供论文发表和同声传译同声传译设备等Google排名服务，专注于企业的SEO优化培训！

  Posted by: jlj714 on October 21, 2007 at 08:12 PM

• 网站优化-搜索引擎优化-Google排名-Google左侧排名-SEO

  Posted by: jlj714 on October 21, 2007 at 08:13 PM

• 博澳流水线以丰富的经验和先进的计算机辅助设计生产线来满足客户的要求，设计出合理的工业流水线.
  

  Posted by: sunhuanjun on November 18, 2007 at 11:05 PM

This work is licensed under a Creative Commons License.