John Reynolds's Blog

Open software pragmatism - Free (as in beer) isn't the point

Back in the mid-90's I was working on the user interface for a Windows desktop application that included a fairly sophisticated cross between a table and a tree component. We wanted to display a multi-column table of assets where each row was actually a summary of information; for example one column was the total value of the asset. Each row of the table could be "expanded" to list the details of all the transactions that were "rolled up" into the summary. You see similar components all over the place today.

Fortunately, even in the 90's we did not have to write this component from scratch. A commercial table/tree component was available (for a reasonable price), and we were able to purchase a source-code version that allowed us to modify the code to our hearts content. One huge advantage to having the source code of the table/tree component was our abilitiy to debug problems, and if the problem turned out to be in the table/tree component itself, we could actually fix the bug instead of having to work-around it.

The downside to having the source code came whenever an updated version of the table/tree was released. If there was a feature in the new version that we wanted to use, we'd have to reapply all of the enhancements that we'd made to the previouse version, and more annoyingly we would have to figure out if the bugs that we'd found in the older versions had been fixed (if not, we'd have to hope that our old fixes could be applied to the newer code base).

As I mentioned before, this was in the mid-90's. The Free Software Foundation was alive and well, but very focused on Gnu's Not Unix. I was working in the Wonderful World of Windows, where Open Source was yet to gain a foothold, and everything cost money: You paid for development tools, you paid for libraries and components, and you certainly paid for source code. Perhaps because of my youth and enthusiasm, paying for stuff didn't bother me very much. I remember that I would often buy a development tool*** that I wanted to use ... usually from Borland because they had great tools for a reasonable price... and then talk my boss into letting me use these tools at work. If I was lucky, I would get reimbursed, but that didn't very happen often, and I usually didn't mind. (*** Note that free development tools such as the Gnu C++ compiler and Emacs were available on Window at the time, but they just didn't have the polish of the tools that you paid for)

Paying For Stuff wasn't annoying; Persistently Unfixed Bugs were annoying. I remember being really pissed with the vendor of the commercial table/tree component because there was not an effective process for submitting bug fixes to the vendor (It was understandable that a reported bug might not make it into the next release... but not when you had sent them the fix). My colleagues and I became very intimate with the internal details of this component, because we were stressing its functionality a bit farther than it could go... and we often found (and fixed) bugs in the component long before the vendor knew they existed. Applying these patches over and over again to subsequent releases of the code base really "chapped my hide".

Life here in the 21st Century has certainly changed with respect to Paying For Stuff, especially with respect to source code. I've done no real survey, but I would be willing to bet that the source code for most of the frameworks, libraries and components that developers rely on today are readily available. We simply don't tolerate hidden code anymore: If I can't see your code, I'll use somebody else's code that I can see (whether or not I have any intention at all of actually looking at the code).

As the last bastions of "closed code" fall, such as Sun ovecoming its earlier reluctance to "Open Source" Java, I want to shout out a reminder:

Free (as in beer) isn't the point

Go back with me to the mid-90's and revisit the issues that we had with that table/tree code that we'd purchased: We were pissed that bugs that we had fixed ourselves never made it back into the official code base, and that we'd sometimes have to significantly mangle the code to add in our own enhancements. If there had been an effective process for getting bug fixes back into the code base, and if there had been an effective process for refactoring the code base to enable end-user enhancements, we'd have been tickled pink.

The same is true today. Allow us to get our fixes back into the code base, and pay attention to our requests for refactoring and API changes, or it's not going to make much difference that the code is now "Open Source".

Comments

• Wow ... so, being free from proprietary software that might not be there tomorrow, as it happened to VB6 ... isn't the point. Being free to say NO to DRM and Trusted Computing and being able to control the platform you make your money on ... isn't the point.
   
  Being in control ... I guess that doesn't matter much, especially for SUN.
   
  Why does SUN even bother to open Java ?

  Could it be that the whole Java community is placing their trust in a proprietary platform that can be exterminated by SUN ?
   
  Ah well, I was wondering when this movement will became "monetized" by corporations and be covered in shit.
  

  Posted by: bonefry on September 02, 2006 at 05:51 PM

• Bonefry,In an attempt to make you happier, I modified my title to be more explicit: "Free (as in beer) isn't the point"That's what I had in mind.-JohnR

  Posted by: johnreynolds on September 02, 2006 at 06:32 PM

• 
  Great thoughts, John. The way i see it, there are two "communities" around opensource software, one is of users, who wish to contribute bug reports and suggestions for improvement, and the other is developers, who wish to contribute fixes, eg. because they need the fix before the vendor does.

  The community around Java is by definition, one of knowledgeable developers, who get to look at the java libraries all the time (eg. we Alt-G into the libraries to check arguments and what-not, read the javadocs, and see what that method call does and means). So in this case, the ratio of "developer community" to "user community" ie. bug fixes to bug reports, is potentially higher than in other end-user oriented opensource projects.
  
  I guess Sun needs some Linus'es and Morton's, whose job is to assess incoming fixes, and submitters, and politely reject fixes with constructive suggestions, so as to be accepted next time. It might be good if submitters are required to adopt a TDD-ish approach, where first one writes a test (which fails because of the bug), then produces a fix, does full regression testing, and then submits the new test together with its fix, and regression test report.

  Having said that, the problem with fixes, is that there might be some big projects out there that have workarounds for those bugs (by a long lost programmer), and to those projects, the fix is a new bug that breaks their software unexpectedly.

  By the way, I cynically suggested in "CDDL'ing up to Sun" that "Opensource promotes common development, where the idea is to engender a community of contributors. But maybe in some cases it's more about distribution. And in others, it's a marketing gimmick. And in most cases, it's some combination of all of the above!" I'm sure that in the Java case, the "common development" motive will weigh in quite highly, given that we "users" are all Java developers :)
  

  Posted by: evanx on September 03, 2006 at 01:31 AM

• personally, I couldn't care less. In fact I prefer a system where there is a single entity that's responsible for (and the only one allowed to) releasing the product.
  It prevents others from introducing bugs and destroying functionality in attempts to "improve" the product. It also prevents people from presenting me with a problem when their customised versions are deployed on the systems of my users and cause compatibility problems with the version I decided to use.
  Most OS products have little or no safeguards against that, or else use licenses that make using the product impossible without resorting to releasing your own product under that same license (GPL...).
  Allow us to get our fixes back into the code base, and pay attention to our requests for refactoring and API changes, or it's not going to make much difference that the code is now "Open Source".
  And what if those "fixes" actually destroy functionality elsewhere in the product, in an area you just happen to not use? That's a prime reason for "fixes" to be rejected. That and people submitting "improvements" that aren't. Maybe you are skilled enough to not make such "mistakes" (though often they aren't, but deliberate attempts to steer a project in a new direction) but many others aren't.
  I've reviewed bug reports from customers, at one point well over half were actually RFCs masked as bug reports. Had that been an open source project and those "bug" reports "patches" that product would have changed dramatically from what it was designed to be, to the detriment of most of the users.
  Sure having the code of something is nice, but I don't need or even want the option of releasing that code under my own name as a fork, which is what most open source advocates see as the core definition of "open source".

  Posted by: jwenting on September 03, 2006 at 01:38 AM

• Jwenting,I know exactly what you mean about RFCs masquerading as bugs... but there are also those "real" bugs that ought to be stomped out. I don't advocate giving everyone commit privledges to the code base, I advocate an effective process for accepting candidate bug fixes, and applying them if they're up to snuff.-JohnR

  Posted by: johnreynolds on September 03, 2006 at 04:22 PM

• But that would effectively put you back at square one. If you don't agree with a decision to reject your bugfix (maybe because it would have negative impact elsewhere in the system and something else was done instead to fix it) you'd still have to either keep patching your own code or change it with every release to incorporate changes in the common codebase.
  In fact unless you stick with a single release of a product (whether open source or not, but more likely with open source as there seems to be far less control there over things like backwards compatibility testing) you're always going to have to do that unless you create an internal fork of the product as a separate project and use only that.
  Take Jakarta commons as a case in point. At one point last year they released what was supposed to be a minor release but that minor release did involve a complete change of major public APIs. Or take HSQL, which changed its package structure between one release and the next so no existing code would run or compile anymore that made use of it.

  Posted by: jwenting on September 05, 2006 at 05:57 AM

• And no, that doesn't mean you should not use external libraries. But it does mean you should be extremely critical about which you use and as much as possible limit their impact on your system by for example using wrappers around them.And of course you don't have to install every update for them that becomes available, in fact you may well decide on a specific version and stick with it unless and until it becomes necessary to use some new functionality that your adopted version doesn't (yet) support.

  Posted by: jwenting on September 05, 2006 at 06:00 AM

• @jwenting:

  Looking into projects like Apache, it seems your fear is misplaced. Apache and its Jakarta sub-projects have a very high level of quality and in most cases, developers are quite conservative in introducing changes. If an Open Source project is maintained by responsible developers, I think the issues that you have mentioned above appears less of an issue.
  

  Posted by: vhi on September 05, 2006 at 06:57 AM

• great post john.

  Posted by: ilazarte on September 05, 2006 at 07:23 AM

• 
  That's one good viewpoint and hopefully well taken.

  Here's another. My thoughts are that Java is better than C for the basic application and library infrastructure. It's also well-known and widespread. But it won't ever replace C in the low-level open source infrastructure (think Gnome, think Apache, think Subversion, think Mozilla, ...) if the Java that people use isn't LGPL/GPL/ASF compatible.

  Posted by: tompalmer on September 05, 2006 at 10:25 AM

• Tom,Thanks for the comment... The point I attempted to make is that even if the Java license is Gnu compatible, it won't be really "open" unless the process for bug fixes, enhancements, etc. is open (a better word is probably inclusive).If the only option that a developer has for improving the project is to fork it, then you haven't really met your aim.In my opinion, this is not a problem for Java... the processes are already very inclusive... but there are many "corporate open source" projects out there with poorer track records.
  -JohnR

  Posted by: johnreynolds on September 05, 2006 at 11:05 AM

• whi, what I described is exactly what happened at Jakarta Commons...
  It broke more than a few applications which forced a lot of people to either stick with older versions or invest a lot of effort (read money) into changing all their code to use the new methods and classes.
  Tom, why the heck is Java useless for lowlevel programming UNLESS it changes to GPL? Last I checked licenses had nothing to do with suitability of a language for a technical purpose. Unless of course you're one of the religious fanatics who don't want to use anything unless it's GPL.

  Posted by: jwenting on September 05, 2006 at 01:04 PM

• 
  jwenting, what I mean is that there are _existing important projects_ that don't use Java but possibly might change someday (even if they don't want to immediately) if Java is available in a compatible license.

  JohnR, thanks for the additional clarification. I do understand your point, and I'm also happy that Sun does okay so far and seems intent on improving. I think one risk or strength (depending on your perspective) for Java is that with so much specified by JCP, you can't just contribute new features. It needs to go through a lot of process. But hopefully things will get well-enough oiled that at least quality patches for outright bugs will be easy to get put in.

  Posted by: tompalmer on September 05, 2006 at 02:11 PM

• Oh, and I think open source Java is good for other reasons (since I'm generally an open source fan), too, but I'll save those for another day.

  Posted by: tompalmer on September 05, 2006 at 02:15 PM

• I blogged about Open Development and distributed version control systems, in response to this post.

  Posted by: brondsem on September 07, 2006 at 10:13 AM

This work is licensed under a Creative Commons License.