Rich Burridge's Blog

Extensible Open Source projects.

Quite often an open source project is started because somebody wanted some functionality that didn't already exist (or that they weren't aware of). A small project is released and it will either flourish or languish depending upon whether others find it useful or not.

I've seen small one time hacks even become released products from commercial companies. Once such a one-off hack becomes a product, then it needs to be supported which can be quite painful if the initial hack wasn't pretty.

A better approach would have been to properly design the application, get potential user requirements, test prototypes with real users, create a functional specification, then work towards an implementation (making sure that everything is fully documented). A lot of work, that not everybody wants to do.

Successful projects have typically taken those steps and more.

More and more I'm seeing projects that provide some kind of extensible functionality. One such approach is with plugins. For example, vim provides this ability to allow you to automatically include scripts that get loaded when the editor starts. Gimp provides a plugin capability to allow you to have extra image manipulation functionality beyond what exists by default. There are lots of others.

With Java apps, a nice way to provide extensibility is to dynamically load certain classes depending upon what you want to do. I used this approach to dynamically load the transport layer code in JSDT, a toolkit for writing collaborative applications.

There are other approaches; graphical and audio codecs, web browser plugins, shared libraries etc.

There is a catch in that this kind of functionality isn't always the most straight forward approach to doing something if you are just focused on trying to achieve a simple task. Sometimes programmers adopt the "quick'n'dirty" approach rather than consider the greater good. As projects become popular, then functionality like this can be added. There is an old adage that says that a program isn't any good until it's been rewritten 2-3 times and that you should always throw the first one away. Perhaps a better approach would be to take more time and be a little more thorough in the way that that first version is created.

Comments

• requirements gathering for quick hacks???
  While gathering requirements and doing interviews with intended users, plus designing for being future-proof are indeed good ways to make an application that can be maintained and expanded (plus is marketable) it's hardly what can be expected of the quick hacks and small tools you describe.

  A quick hack that gets released as a product was never intended for release.
  It was intended as a one-off for someone to use internally that someone else liked and started using, which others saw him use and liked, etc. etc. until someone in marketing sees it and likes it.
  Had that quick hack had to go through a formal requirements gathering and design phase it would never have seen the light of day as that would have taken too much time to help with the initial requirements of the person writing it who needed a quick solution to a (seemingly) unique problem.
  The full path would likely have taken months and required a complete project with budget meetings, review meetings, etc. etc. which would likely never have been approved (let alone that it would not yield the application in the timeframe the writer needed it).

  Posted by: jwenting on May 07, 2004 at 02:04 AM

• A couple of references
  Luis Villa posted a
  link
  or
  two
  to interesting papers with extensibility in mind.

  Posted by: richburridge on May 07, 2004 at 12:08 PM

• Adage
  | There is an old adage that says that a program isn't
  | any good until it's been rewritten 2-3 times and that
  | you should always throw the first one away. Perhaps
  | a better approach would be to take more time and be
  | a little more thorough in the way that that first
  | version is created.

  If you say that, your obviously not really a programmer.

  If you are a manager you should also know better; designing an application takes a lot more time then coding it. Doing research to what the requirements would actually be is cheapest by asking someone who is knowledgable in the field.
  If nobody is knowledgable in that field the only way to get knowledge enough to make a good design is to actually create an application that does what you want. Naturally during this design you start to learn what the requirements are and in the end your application may be finished; but its not really 'designed' and needs heavy refacturing.
  At this point the developer is for the first time fully aware of the requirements. THEN he can start to implement the application.

  This story is naturally a bit black/white as the eXtremeProgramming concepts make the process differ per project.

  A programmer proposing a rewrite (of his own code) is always good for the long term maintainability of the project. If you ever did, you'd agree with me.

  Posted by: zander on May 08, 2004 at 01:35 AM

• requirements gathering for quick hacks???
  "A quick hack that gets released as a product was never intended for release."

  Agreed. Quick hacks are fine and they have there place, but they should never be released as products until/unless they have been through a product methodology. As you say, that would prevent most of them seeing the light of day.

  Posted by: richburridge on May 08, 2004 at 04:03 AM

• Adage
  I am a professional programmer. Have been for 29 years. I've even been known to think "the developer knows best" at times. Certainly when I was doing some projects because they were fun and that's what I wanted to do. That fine but...

  Once a project is intended to be a product, that changes. You do need to take into consideration your customer requirements. Find out what they really want. Don't guess or make assumptions. You do need to involve quality HCI people in the process and let them design how the product looks and feels. That's their job. They hopefully understand this stuff better than the developers do, otherwise it shouldn't be their job. This is where the second or third version occurs. It's recommended that you also get the architecture of your project peer reviewed. Others see problems that you might never of thought about.

  Posted by: richburridge on May 08, 2004 at 04:14 AM

• Adage
  HCI: If you find yourself doing 2nd or 3th revisions because of HCI matters you should revise your programmers workflows.
  Our HCI experts tweak the GUI with a graphical GUI designer and most of the time the programmer only needs to spent 10 minutes to incorporate those changes afterwards. http;//uic.sf.net is what makes that possible.

  Architecture; thats what I was talking about in the previous post. No need to repeat it. I'll just repeat my pointer to XP.

  You said 'extensible open source products'; I'm not sure where the "Intended to be a product" came from. I interpret the former to be a project that is used by many peers in one way or another. Extended to fit someones needs with functionality that everyone will appreciate. Ant is an example of what I mean.
  The latter (a project that becomes a product) can be quite different since your company has to anticipate features and implement a lot before a customer may consider buying it. Its market driven while the former is driven by direct desires.

  I'm still talking about the former; a project that can grow and many in the end some other company will find it can be used as a basis for a product.

  The fun thing is when one complements the other; Java is perfect for that. Add several building blocks and the end product gets bigger. If one of these building blocks is an extensible open source project, all the better!

  Posted by: zander on May 08, 2004 at 12:23 PM

• requirements gathering for quick hacks???
  Thats just silly, and you are taking away an awful lot of potential products from your boss!

  There are various plugins for a product we make that were quick-hacks at first but were used to convince sales about their potential.
  We ended up reviving the quick-hack project over a year after it was abandoned and removed from CVS. Recompile; re-implement some classes and away you go.

  Don't say it can't be done when people are actually doing it!
  Don't say it shouldn't be done when people are actually profitable by it either :)

  Posted by: zander on May 08, 2004 at 12:27 PM

• requirements gathering for quick hacks???
  If you are saying "Recompile; re-implement some classes and away you go" it looks to me like you aren't providing backward compatibility. That definitely qualifies as a quick hack. I wouldn't classify it as a very supportable product. Perhaps I should have said it shouldn't be done rather than can't be done.

  Posted by: richburridge on May 08, 2004 at 05:30 PM

• requirements gathering for quick hacks???
  re-implementing some classes has no effect on API; so no problems with backwards compatibility.

  Recompiling is a bit deceptive; I use a code-generator that generates GUI code based on project settings; a recompile is needed if the project settings have changed, if only because the newest version of the code-generator works around more Swing bugs.

  I know you meant that it shouldn't be done. And I find that silly because lots of people are doing it, and making a profit of off it.

  If you don't like quick-hacks; you will never create a Swing application since Swing is soo ridden with bugs that you absolutely HAVE to work around problems in a quick-hack kind of way. At least if you don't want to created dozens of new models and widgets from scratch.

  On the backwards compatibility thing: who cares if your customers are using webstart.

  Posted by: zander on May 09, 2004 at 12:07 AM

• Adage
  Thanks for the uic.sf.net pointer. I'll try it next week.

  "You said 'extensible open source products'; I'm not sure where the "Intended to be a product" came from."
  One thing I'm learning with bloging is that I should probably confine a blog entry to a single topic to avoid confusion. This blog entry ready talked about two things:

  quick hacks that become products
  how you can provide more value by making your projects extensible

  
  doing one probably mean you're doing both. Certainly not for quick hacks.

  I'm also trying to be a tad contraversial to force others to state a different opinion. Sometimes old far\h\h\htimers like me are becoming too set in their ways and need to hear a different approach.

  Posted by: richburridge on May 09, 2004 at 08:49 AM

• Adage
  Replying to my own reply. There's a sign I should stop for the day. I wrote:

  "doing one probably mean you're doing both. Certainly not for quick hacks."

  That should have been:

  "doing one probably doesn't mean you're doing both. Certainly not for quick hacks".

  PS: I wish there was a good blog editor that could handle this java.net blog style.

  Posted by: richburridge on May 09, 2004 at 08:53 AM

• requirements gathering for quick hacks???
  If the product is small enough (and most quick hacks are, or are at least small enough parts of a larger whole) supportability is a moot point as the quick hack is small enough to be either inherently stable or relatively quick to swap out for something that better fits the pattern-of-the-day of the company architect if needed.

  Posted by: jwenting on May 10, 2004 at 12:08 AM

• Adage
  you're talking about requirements and customers.
  Quick hacks notoriously arise most often in situations where there are no customers except the person writing the quick hack and the requirements are all in his head.

  To start a formal design phase for the software would mean the development of the project for which the quick hack is being created as a sideshow (which is usually the case) will grind to a halt for possibly months until the (now formalised into a major project) sideshow has been implemented.

  That scenario is leading to disaster.

  Posted by: jwenting on May 10, 2004 at 12:14 AM

This work is licensed under a Creative Commons License.