The Source for Java Technology Collaboration
User: Password:



Kirill Grouchnikov

Kirill Grouchnikov's Blog

Open source - the curse of the abundance

Posted by kirillcool on January 11, 2005 at 03:25 AM | Comments (11)

As this blog shows, almost every imaginable field of the programming has been covered with open source Java projects, all vying for the same #1 spot. It may seem at first that this can only be a good thing, competition driving the creators forward to create better and faster libraries. However, not all is fair in this kingdom.

A recent survey that has been performed on the XML data binding frameworks (available here) has provided me with some interesting insights. In this survey, nine different libraries have been tested for time and memory performance on the same XML input (and its Java representation). The biggest problem? Had to write code for each one of them from the scratch. Even though the JAXB API has been available from 2001, the frameworks have not adopted the proposed approach (each one because of its reasons, i'm sure). The result - each marshalling and unmarshalling class has its unique name, unique signatures for its functions and so on. The bottom line for a developer that wishes to switch implementation - no can do.

This seemingly innocent approach of reinventing the wheel (even when the original one is good enough and the only problem with it is that it wasn't invented by you) effectively prevents a potentially successful framework from taking the market lead. Suppose tomorrow a great library comes out, all blazing with performance, with full support of XSD, DTD and Relax. Who will use it? Only the new projects. The cost of switching to a new library and rewriting your code will be too prohibitive for any existing midscale up project.

In addition, this also results in libraries that have not been tested thoroughly in production environments. A recent example of this has been discussed at this entry. This particular class has test functions, they just don't cover the buggy line. It has been there since 0.6.0 version (up until the current 0.6.4). The availability of the source code and the testing by the author apparently have not been enough. The data structure market has been divided by so many players that a new player just goes unnoticed (except by Google).

Lack of coordination between different libraries (even on the most basic interface level) doesn't allow me as the end user to test the libraries by simply switching the implementation in the configuration file. The result - a really good library is sitting somewhere there languishing.

Enjoy
Kirill

Bookmark blog post: del.icio.us del.icio.us Digg Digg DZone DZone Furl Furl Reddit Reddit
Comments
Comments are listed in date ascending order (oldest first) | Post Comment

  • Hi,
    I have a quite successfull project under my lead and have been doing it for 3 years now. Quite often I see a new product announcement which tries to fill a niche that I feel is already covered in my project.
    I usually send a friendly email trying to find out why a new project has been started and what they feel they should tell people if those people have to choose between different open source projects.
    The answer. most of the time, is that they have no idea there already is a project which kinda does something similar to what they do.

    In the end I always just delve into their sourcetree to see if they had ideas that were particularly good to integrate into mine. I have never been able to convince someone to join forces. I only partly blame that on my social skills, though. The not-invented-here is getting in the way much more often.

    I feel that your complaint that switching should be possible is misguided since cooperation could not happen without knowledge of each others existence, and respect for each others opinions. Something rarely seen in open source Java applications.

    Posted by: zander on January 11, 2005 at 04:58 AM

  • Hmmm... what does abundance of choice has to do with open source?!?!?


    You could run the exact same bogus headline against anything:


    Proprietary software - the curse of abundance


    Please consult Adam Smith for a rational.

    Duh.

    Posted by: zoe_info on January 11, 2005 at 05:04 AM

  • Is this more to your liking:


    The Cognitive Style of Java

    Posted by: zoe_info on January 11, 2005 at 05:26 AM

  • In the case you mention, I think weak standards are the real culprit. I do however agree with what you're saying as a whole. After all, how many linux distros are really needed?

    Frameworks such as Spring, Pico, the hundereds of web frameworks, and more recently rich client frameworks, etc.... emerge when colloaboration and consolidation of the projects would be more benefical to the industry instead of competing standards.

    Open Source solutions that comply with strict standards can be just as succesful as solutions provided by commercial offerings. Well defined and ridged standards are the key.

    Posted by: rabbe on January 11, 2005 at 05:39 AM

  • zoe_info

    Blindly attacking proprietary software is no argument in this case. I prefer the argumented response of Zander that lines out the problems he faces in the open source world. The headline here doesn't imply that the problem lies only with open source. This problem, however, must be faced and disputed, but not with catch-phrases like "Microsoft is evil", let's depict Bill Gates as Stalin.

    Posted by: kirillcool on January 11, 2005 at 05:43 AM

  • The headline here doesn't imply that the problem lies only with open source.


    Then dutifully amend your headline by perhaps dropping open source from it.


    This problem, however, must be faced and disputed


    What problem? That people are free to do whatever they please, highbrow notwithstanding?

    Posted by: zoe_info on January 11, 2005 at 05:58 AM

  • Well Said! I agree with most of what you said. As far as I think there needs to be competetion and compatibility in Open Source frameworks, toolkits and applications. When some one is starting a new open source project, they should look for other similar products and perform a clean analysis from the business, system and product perspective. Websites like http://www.manageability.org/ and http://www.osdir.com are good places to search the open source tools.

    Like zander said, some of the open source developers hardly have respect for each other - the reason I think is that it is their way of expressing how their product is different from some body else's. A good example is Cedric (This is not a personal attack but anyone who reads his blog forms an opinion about his respect for JUnit developers - Eric Gamma and Kent Beck. A link is not provided not to propagate the madness).

    Another problem that you commonly find as open source development is its methodology -- "Let us throw the crap on the wall, and see what sticks." You can find this in some of the Apache projects!

    Posted by: chakrayadavalli on January 11, 2005 at 09:11 AM

  • Why don't we all wear the same clothes, drive the same car, have one world government, and so on? Heterogeneity is good - not perfect, good.

    Heterogeneity leads to innovation, which leads to new capabilities. New capabilities help us solve problems and achieve goals.

    Typically too, what seems to be two equivalent APIs (or components, or…) is not. The inputs and outputs may be the same; however, performance is usually not. Why don't we parse XML with only one method? It is because some methods are superior depending on the size of the document and/or the intent of the parse function in the application. Tradeoffs are a pervasive characteristic to life.

    At a higher level, heterogeneity is exciting in the context of SOA. I am designing a Semantic Service Oriented Architecture (SSOA). In SSOA, services are exposed as Java interfaces using Jini technology, and the services are semantically described using Semantic Web technologies. This approach provides several benefits, including the ability to decouple function from implementation, and allowing automatic service discovery, and invocation.

    Tradeoffs and “non-equivalence” apply from the most atomic levels (e.g. UTF) to the large, conceptual spaces (e.g. SOA).

    So I say be creative and keep it coming! We can handle it.

    If you want to know more about SSOA email me at sgchance@comcast.net

    Thanks! Sam

    Posted by: sgchance on January 11, 2005 at 10:06 AM

  • And it's not just libraries. Look at the thousands of frameworks that appear out of nowhere and go nowhere either.
    Hardly anyone uses them except their authors. All OS addicts roll their own, everyone else does so too or goes for a well established and supported system instead.

    Posted by: jwenting on January 12, 2005 at 12:48 AM

  • I suppose that some kind of meta API exposing common functionality would solve your problem. One could program to this interface and each implementation of the interface would encapsulate one XML parser. Thus, it would be easy to switch to another XML parser later in the development cycle if the first choice turned out to be a bottleneck or error prone.

    Sure, writing such a meta API would take an afternoon but there would not be much maintenance from that point on. Might be an open source project in its own right. Anyone?

    Posted by: bridgehajen on February 18, 2005 at 03:55 AM

  • I suppose that some kind of meta API exposing common functionality would solve your problem. One could program to this interface and each implementation of the interface would encapsulate one XML parser. Thus, it would be easy to switch to another XML parser later in the development cycle if the first choice turned out to be a bottleneck or error prone.

    Sure, writing such a meta API would take an afternoon but there would not be much maintenance from that point on. Might be an open source project in its own right. Anyone?

    Posted by: bridgehajen on February 18, 2005 at 03:55 AM





Powered by
Movable Type 3.01D
 Feed java.net RSS Feeds