Felipe Gaucho's Blog

Using elephant bullets to kill small ants

Few months ago we started two very similar projects in my daylight job. Both projects were WebStart applications based on simple business rules with a lot data validation and a fancy GUI requirement - almost all Swing components were used to produce the visual aspect our client had requested. Those projects shared the same life-cycle, under the same management model but with different teams. A project was carried out by a junior team while the other was formed by experienced programmers. Few weeks after the beggining, we noticed a critical difference between the projects: its sizes - both projects had a very similar number of code lines but the former was much heavier than the latter. The junior team produced their system based on third party libraries, while the experienced programmers provided a lightweight solution. Evaluating all the development process we observed a very common scene in the software industry: velocity versus reasoning. While the experienced programmers designed the software based on their solid background about the Java API, the newbies relied on Google to ask how to do their duties. Many tasks were solved through a download of a third party library. JDom, Log4J and many other JARs were dropped in the lib folder without much care concerning the impact in the project size. The tasks were quickly developed and any new issue was fixed through a new Google search. Other interesting aspect is that those libraries had also their third party dependencies, i.e., a chain reaction.

You may conclude the problem was in the junior team, the people was unskilled for the project. But ask yourself how many libraries you adopted in your last project and also if all those libraries were really necessary.

Few years ago people produced bank applications with less than 1Mb of memory (do you remember ambar screens?). Nowadays, any computer without 1Gb RAM seems useless for the developers. Part of this complexity comes from the user demand - friendly GUIs, robust systems to inapt users and the sophisticated technology and tools. Other aspect is about the interoperability among systems that pushes the simplicity away (SOA? EJB?). It is a natural motion in the software industry and people are not concerned about that - but there is the developers side. If one starts a simple project without thinking about its size, a system could depends on 10 Mb or more just to read mails and save files, and that's seems to be a next quality concern.

When you think about software quality, do you include the size of the project in your evaluation? And what about dependencies?

Ok, I'm not suggesting you to make all the hard working from the raw concepts and using only the Java API - but I feel sometimes we are loosing the sense of equilibrium in that question. My suggestion: to adopt good libraries just after comparing their features with the JRE API and also after a brief evaluation about their benefits in the project life-cycle, including the future perspectives of both projects - your project and the exogenous libraries.

Let's think about that ....

Comments

• I think download size should be almost the smallest concern of any developer. Most people have fast connections and I think most people will gladly download a 10MB installer. Anyway, if we all use JWS or JWS-based installers, we can save download time by sharing jars through shared jnlp library descriptors.

  Posted by: keithkml on October 08, 2005 at 01:27 PM

• What do you mean by 'heavier'? If you meant the application's performance is worst, maybe it isn't related to the third-party libraries at all. Most performance problems with Swing applications are not real 'performance' problems, but 'perceived performance' problems. Things like long-running tasks in the event thread, and creating complex components every time instead of reusing them when possible, cause a horrible impression when using the application.

  And by using third-party frameworks and components, you can save development time - by avoiding reinventing the wheel. Well, this phrase is a little overused, but it still true for many cases.

  But it has another advantage: you raises the chances of finding someone capable of doing the maintenance after you are gone. For example, if you use JDNC components in your applications, someone who already knows JDNC may be capable of understand your code right away, instead of trying to figure out how a bunch of custom components work.

  For the JDOM part... c'mon, the DOM API sucks, it's a fact. If you don't use JDOM or DOM4J, probably you'll end up writing your own wrapper. And Log4J could be abstracted away by using commons-logging (even though many of you may hate it), and substituted by java.util.logging if it's size became a problem.

  If the bigger download size is your problem, you may try to remove unused classes from the third-party libraries, probably most of them aren't used. If you are worried by the time it'll take to load all those jars, well, most JVMs load classes on demand, so probably it's not a problem too. If everything else fails, just put up a nice splash screen ;)

  Posted by: ronaldtm on October 08, 2005 at 07:23 PM

• What I mean is, the junior team did not produced bad GUI because they used third-party libraries, but because they were a junior team.

  Posted by: ronaldtm on October 08, 2005 at 08:43 PM

• (to me) Stupid, read the text completely before answer!
  Sorry for answering the wrong question...

  Back to the topic: I agree with you in that we shouldn't add a third-party library if it doesn't add any value to the project. But my main concern is not the size of the package, or if the software isn't performant (like I said before, it's probably not caused by the libraries, but by poor programming). The real problem is complexity.

  I said that by using well known libraries, you can leverage common knowledge from developers, facilitating maintenance. But at the same time, if the library just wasn't necessary from the beginning, it'll just add unnecessary complexity. The whole point of using a third-party library is not to remove complexity, but to add less complexity than if you had to implement everything from scratch. If there shouldn't be anything to implement from scratch (I mean, you don't need the library), you are only adding unnecessary complexity...

  Posted by: ronaldtm on October 08, 2005 at 09:38 PM

• ok, thank you all for provide me different points of view.Still thinking about WebStart tranfer costs: we had discussed about the need of a JAR thinner, i.e., a tools which check all system dependencies and do remove the useless classes from the deployable JARs. It can be a difficult task, due to runtime dependencies. The original inspiration of my blog entry: some of our customers are still using Dial Up connections (do you remember that?) - unfortunately low bandwidth connections are a reality in many countries.

  Posted by: felipegaucho on October 09, 2005 at 09:15 AM

• Regarding your need for a "JAR Thinner", I have used an obfuscator before to do exactly this. I think I used yGuard, and it worked pretty well. It will start to get tricky if your app (or its libraries) involve a lot of reflection, though.

  Besides that, seeing as you are using WebStart, you can Pack200 your JARs which makes them amazingly small compared to their natural size. See this article if you need to do it for Apache: http://joust.kano.net/weblog/archive/2004/10/16/pack200-on-apache-web-server/

  Posted by: grlea on October 09, 2005 at 05:56 PM

• Philosophically speaking, in scenarios such as yours, I think the advantage provided by using a library needs to be weighed against its size. Dom4J is an excellent library and very useful. But if there's only one place in your code where you need to parse a document, it's hard to justify including a 1/2 meg JAR. On the other hand, if your application is some kind of XML manipulation tool and you're parsing and manipulating documents all over the place, the 1/2 meg cost is probably worth the development time you'll save.

  This principle can be applied to any library. Sometimes there will be alternatives that might better suit the situation. I believe Log4J is a complex behemoth that provides way more than what most applications need, so I created Simple Log as a small, simple alternative. Sometimes when you need a simple alternative, it can be pretty easy to write one yourself, and the resulting code, because it's much more specific to your needs, will be much smaller than any generic library could be.

  Graham.

  Posted by: grlea on October 09, 2005 at 06:31 PM

• The main problem with custom built software is indeed runtime performance and memory footprint, not disk footprint.

  With shrinkwrap software disk footprint becomes a problem by proxy because of the ever increasing feature bloat of many applications which leads to gigabytes being installed to harddisks to render functionality which 99% of users never need.

  The first (runtime performance and memory footprint) is caused in no small part by laziness, but time pressure also plays a part.
  Let me explain... Programmers become lazy as hardware becomes faster and memory increases. No longer do they need to pack bits into bytes to fit into the smallest memory space, and as a result compiler creators no longer create fast functions to achieve this.
  No longer do they need to count the CPU cycles because every cycle wasted costs money and makes the application noticably slower. As a result profiling has been relegated from a being priority during application development to a rather obscure afterthought done only when customers complain.
  That's the laziness, brought on by ever increasing hardware power making it possible to create less optimised software which runs at the same speed as the old stuff it replaces. As customers are used to that speed they won't notice their new software isn't any faster despite the higher potential of their hardware.
  This has been going on for so long now it's become an established fact of life that a 50% faster machine is needed every 2 years to retain the same runtime performance of new applications, which is an insanity.
  The other factor is time pressure during development. Because this same attitude has also penetrated (possibly to an even greater extent) the business side of our companies, we are given ever less time to ensure decent performance. As more and more functionality needs to be created "lean and mean" (meaning quick and dirty of course) in any given time period, the time available for proper design and optimisation is reduced more and more. This leads to things being created on the fly that, had they been properly thought out earlier, would have been smaller and better performant.
  Of course the cynist in me sees another reason for businesspeople to work like this and that's that many of our companies not only supply the software our customers use but also at least get a decent percentage on hardware sales. Therefore being able to tell the customer his hardware is getting old and really needs replacement with a new server and workstations to make optimal use of the new software product is a nice added bonus, and those customers who defer will come back for that hardware anyway once they start slowing down in their work.

  Posted by: jwenting on October 10, 2005 at 04:20 AM

• Clearly download size is an issue for his users. You must not have done much downloading on a 33.6Kbps modem. Usually, I got about 3K/s downloads, so a 1MB file could take more than 5 minutes. If your webstart jars are 10MB in size, how many users will be willing to wait an hour to download?

  Yes, you gain maintenance from utilizing 3rd party libraries, but you can't forget that your customers don't care about that. Yes, it makes our jobs easier, but without the users we wouldn't have jobs at all. And if this tool takes too long to download, his customers might find a smaller one.

  Posted by: heaththegreat on October 10, 2005 at 07:47 AM

This work is licensed under a Creative Commons License.