Looking at 'findbugs'
The leader of the Findbugs project, Bill Pugh visited Sun this week. He gave talks to various teams about the tool, showing its usefulness. One of the talks was to the management of my team, the Java Quality Engineering organization.
It's a real interesting tool, doing what's called static analysis. If you haven't read about it, the basic technique is to scan through the classfiles (not source code) looking for patterns that indicate bugs. For example the tool identifies somewhere around 10 places in the JDK source where, if that code section is executed, the JDK will enter an infinite loop. Hurm.
It's an interesting tool, and Dr. Pugh's talk shows a great deal of real bugs the tool catches. It seems what he showed sure caught the attention of my management team, as we are very interested in how we can apply it in testing Java.
Unfortunately the tool doesn't generate test cases, it simply shows potential problems. In my career I'd worked as a developer for 8+ years before joining the Java Quality team, and I know very well that developers time is valuable. Many of the reports this tool gives the developer is going to ask "is it worth my time to fix this", and try to brush it off as happening in rare situations.
I wanted to see for myself how useful the tool is, and I tried it on a project I'd written over the last year. The project had 157 classes, so it's not terribly big, and it's based on the Jemmy source code. It found around 120 "bugs", and gave a bug percentage of 75% (or so).
However I have a hard time classifying most of the reported things as bugs. For example in several cases I called a method that returned an Object, which I stored in a local variable. I have some commented out code which printed the Object, and otherwise the Object is going unused. Findbugs reported that as a potential problem, which it is since sometimes you mean to do something with such returned objects. But then what, either I get the warning for not doing anything with the Object, or I don't store the returned value in anything, in which case findbugs is going to warn me about an ignored return value.
Since this is java5, there ought to be an annotation to apply which will make findbugs not warn about this.
Another report was of some instances of inner class usage where findbugs recommended I use a static inner class. Hurm, I'm not clear on when it's good to use a static inner class. This points to a major feature request I'd like to see in findbugs - namely - for each warning type, there ought to be available a bit of explanation about the warning, some examples that exemplify what's being warned, and some examples of how to fix the problem. Maybe this warning about needing to make the inner classes static is important, but I don't know why it's important, and I'd love to learn more.
Of the 120 or so problems there were 4 instances where I felt it was a real problem that justified being fixed. Three of them were failing to close a stream, and the fourth was a mistake in initializing a local variable in an inner class. That fourth issue would have led to some confusing behavior.
I think findbugs is pretty useful. This despite the high ratio of nattering issues it reported. See, it's possible to easily filter out reported issues so you can focus on what you really want to look at. Plus, as you use it over history findbugs remembers things it has warned about in the past, and won't warn you about them again.