Open source quality organizations
Today I spent a few hours looking at the presence of quality organizations in different open source projects.
Typically you think of an open source project has being some software developers scratching an itch, and so the processes circle around the software developers. So I expected to find little presence of quality organizations within the various open source projects.
The results of my search surprised me. In some instances anyway. Most of the testing in quality organizations seems to be unit tests written by the developers. While unit tests is better than no tests at all, unit tests have a limited focus. How do you know the whole thing is gonna work well if you only test the parts?
One project that stood out is netbeans. They've not only got a full time QA team, all their test plans are right there on their web site. Some projects (e.g. Eclipse) had zero presence of testing on the web site. I'm sure these projects have tests and test activities, and I'm talking about the visibility of the quality process.
Why is this important? Quality is important! If the open source community were to shirk on quality, the perception of the software will suffer as will acceptance. I remember reading assertions that in some open source projects there's many eyeballs looking at and using the software daily, and that means daily adhoc testing. Well, okay, that sounds reasonable, but is that what you want to rely on? In commercial products you have organized test plans, dedicated test teams, regular testing, etc.
One truism widely noted is that it's a lot cheaper for the developer to find a bug the day after they write it, than it is for the bug to be found out in the field. First, the bug found in the field diminishes the perception of the product, and secondly the bug found in the field takes a long process to raise the attention of a developer so s/he can fix the problem. The next most cheapest place to find the bug is in the weekly testing by the dedicated QA team. And when the organization is being run on a shoestring because there's no money flowing through the organization - cost is very important.
In any case that (whether individual open source projects are having strong QA teams or not) isn't the axe I'm looking to grind.
Instead what I'm looking at is what openness might mean to the Java SE quality team. Today's activity was a kind of literature review, looking to see what models I could find in the open source projects. Oh, and before you readers take this to mean I'm working on open-sourcing Java, no, that's not it ... there is a process working its way through the Java SE team to create more openness. You'll have seen earlier results on this such as the stuff we publish at http://jdk.dev.java.net/. All that's going on is I'm putting together ideas. I'm interested in hearing yours.