John D. Mitchell's Blog

Testing Archives

FindBugs in Anger

If you aren't already using Findbugs then hopefully you've at least heard about it by now and have some idea of how useful it can be.

If not, then let me say that FindBugs is a must have tool in the arsenal of any Java developer and any development team that's not using it as part of their regular development practices is incompetent.

Bill Pugh has done a fantastic job making FindBugs a great F/OSS tool which helps detect a large variety of all too common programming mistakes in Java.

You can find an online demo, slides from last year's FindBugs introduction , and can even run FindBugs over the web.

If you aren't yet convinced that FindBugs is really useful, let me point out that I've used FindBugs as an expert witness in cases where outsourcing projects had gone wrong and people were arguing about the quality of the delivered code (among other things). You have been warned. :-)

Go wild!

JaveOne 2007, Java Puzzlers Points Out Problems with Kitchen Sink

Josh Bloch and Neal Gafter have been doing their wonderful Java Puzzlers shtick at JavaOne for the last 4 years. This year, Bill Pugh has stepped in for Neal to keep the puzzlers puzzling.

The puzzles point out all sorts of interesting and exasperating gotchas in the Java language and libraries that linger for ages waiting to humble us all.

This year's set of puzzles include a slew of problems that come as a direct result of the addition of auto-boxing/-unboxing of primitives to/from their object wrapper types (int and Integer, for example). Maybe it's just me but I seem to recall a lot of us who said how bad the consequences would be of adding auto-boxing to Java and yet here they are causing exactly the worst kinds of insidiously non-obvious problems.

The moral of these puzzles isn't merely "be careful of using wrapper types with the ternary operator", but that auto-boxing is evil and should have never been added to the language in the first place.

Of course, it goes without saying that this is just one example of how new "features" added to a mature language are very dangerous. One of the big, currently brewing brouhahas is over the proposed addition of closures to Java. Perhaps folks might want to pause their rampant fervor for a bit and actually look at the history of these attempts and realize just how costly these changes really are. Especially when there are plenty of alternatives like using other languages which have their pet features which target the underlying JVM.

As Josh so eloquently put, "APIs, like diamonds, are forever."

JaveOne 2007, Community One

Sun is, as everybody knows, struggling to get mindshare around their products. This is especially true as they try to get uptake as they open source more of their stuff -- such as Solaris.

Hiring Ian Murdock of Debian fame is a pretty good idea to me. One of the biggest hurdles to (Open) Solaris uptake is the fact that so many things in dealing with Solaris are so annoyingly odd to all the folks who are used to the relatively consistent GNU userland experience and the usable package managers on Linux and *BSD distributions.

Another item that came through over and over again throughout the day was that one of, if not the key reason to use Solaris is DTrace. DTrace is an efficient execution tracing framework and if you haven't used it, you're missing out. Story after story from a wide variety of developers, sys admins, QA folks, etc. touted how using DTrace allowed them to get insight into the actual running of their systems and how big a difference that can make. While it's an open question of whether/when this will make it to Linux, DTrace is already in the next version of OS X and will be in the *BSDs sooner rather than later.

I must say that I was surpised how little I saw emphasizing the coolness of ZFS. It's a modern filesystem designed for the current disk storage and usage reality rather than how things were 20 years ago. Coupling ZFS with Sun's Thumper box is, IMHO, a compelling reason to actually buy Sun hardware. There's no really good filesystems in the open source world if you actually care about your data and want good performance and manageability. ReiserFS is pretty much orphaned and while the ext family are okay for desktop and non-critical servers, they just don't cut it when the data really matters.

Of course, for Java developers, the question is pretty much moot as to whether it's any advantage to go with Linux or Open Solaris. Java runs well on both. It was quite funny to hear some pushback to Greg Luck's (of ehcache) comment that OS doesn't really matter -- just a good JRE implementation. That's just playing out the old Java mantra of "write once, run anywhere" in the real world. Of course, operating system choice does matter to a point -- Greg's own company is an example of moving from ASP.net to Java because of scalability / performance reasons and days vs. months and years of uptime.

For me, I've used all of them for so long that it's mostly just a question of using what works for any given need. I'm hoping that the continued opening up of Solaris will help spur improvements in the Linux world and that many of the things that we love about the OSS operating systems will help improve Solaris so that moving around from one to the other is even easier.

Tests, Specifications, Typing, Oh my!

There's some interesting discussions taking place on the nature of tests. Brian Marick distinguishes between tests as specification vs. tests as examples. Michael Feathers asks if type systems in programming languages are really tests.

Kevin Lawrence (and the Agitar crew) talk about the philosophical contention between the notions of For All and There Exists. That is, the difference in mindset of existentialists vs. universalists. Looking at debates in e.g., the extreme programming world, there's a lot of confusion and arguments back and forth that stem from this constructivist vs. deconstructivist conflict.

The biggest tragedy in these debates, IMHO, is that people on both sides of the fence polarize and calcify in their self-righteous positions. The fact is that we need some amount of both approaches to succeed. For example, writing test-first leads to horrible code if you don't also refactor as you go. Accretive unit tests that aren't themselves refactored leads to big, ugly and unmaintainable test suites. More simply, positivist (aka "garden path") tests must be balanced with deconstructivst (test (to) destruction) tests.

Stepping up a level or three, it's even better if, rather than merely mitigating and ameliorating problems, we change the game such that it's hard/impossible to even articulate bad ways and trivial/easy to articulate the garden paths. That is one of the biggest benefits to taking a linguistic approach to development.

What does 99.999% reliability really mean?

Michael Levin posted a Java coding challenge this morning to write a Java program to show what terms like e.g., 99.999% reliability actually means.

Here's my quick and dirty take (written while watching F1 :-):

/*
 * What does X nines reliability really mean?
 */

import java.io.*;

public class Nines
{
    private static int SPM = 60;
    private static int SPH = SPM * 60;
    private static int SPD = SPH * 24;
    private static int SPY = SPD * 365;

    public static void main (String[] args)
	{
	spit ("Nines of Reliability: (Hours / Minutes / Seconds)\n");
	spit (2, SPY / 100.0);
	spit (3, SPY / 1000.0);
	spit (4, SPY / 10000.0);
	spit (5, SPY / 100000.0);
	spit (6, SPY / 1000000.0);
	spit (7, SPY / 10000000.0);
	}
	
    private static void spit (int numNines, double seconds)
	{
	spit (numNines + " 9's (");
	for (int i = 0; i < numNines; i++)
	    {
	    if (2 == i)
		spit (".");
	    spit ("9");
	    }
	spit ("%) = up to ");
	spit ((seconds / SPH) + "h / ");
	spit ((seconds / SPM) + "m / ");
	spit (seconds + " seconds of downtime per year.\n");
	}
	
    private static void spit (String str)
	{
	System.out.print (str);
	}
}

Which tells us:

Nines of Reliability: (Hours / Minutes / Seconds)
2 9's (99%) = up to 87.6h / 5256.0m / 315360.0 seconds of downtime per year.
3 9's (99.9%) = up to 8.76h / 525.6m / 31536.0 seconds of downtime per year.
4 9's (99.99%) = up to 0.876h / 52.559999999999995m / 3153.6 seconds of downtime per year.
5 9's (99.999%) = up to 0.0876h / 5.256m / 315.36 seconds of downtime per year.
6 9's (99.9999%) = up to 0.00876h / 0.5256000000000001m / 31.536 seconds of downtime per year.
7 9's (99.99999%) = up to 8.76E-4h / 0.05256m / 3.1536 seconds of downtime per year.

So, 5 9's means less than 5 1/2 minutes of downtime per year. Hmm... Now, how long does it take your server just to boot once?

Anatomy of Insanity

CNet reports that Microsoft is offering $5 (yes, 5) for data loss due to it's new AntiSpyware software that's in beta testing. Gee, thanks. That will buy me a cup of coffee so I can calm down after you destroy my data. Yeah, sure.

This is another case of how Microsoft (and so many other organizations) just doesn't understand (or care) how enormous an impact their buggy software has on users. This goes part in parcel with the wonderful example in my old blog entry Anatomy of Insanity? Of course, they will claim that this offer is somehow helpful to the customers but, I must say, it's just plain insulting. Why not try something revolutionary like actually writing high-quality software?

Use less milk?

My little girl has this habit of pouring way too much milk into her bowl of cereal. Then, she whines and complains when we tell her to drink up the extra milk after the cereal is gone so it doesn't go to waste. Yesterday, she got quite snippy when I dared to suggest that she try pouring less milk into the bowl.

Gee, she sounds like a lot of managers and developers of software.

Though, to be more precise, in most software situations, it's not that they pour too much milk but, rather the converse: trying to eat way too much cereal for the given amount of milk. Of course, it gets even worse when we're asked to eat random weeds from the edge of the parking lot instead of cereal because the powers–that–be misheard some "guru" talking about the miraculous recuperative abilities of mountain goats who eat nothing but weeds.

In terms of software estimates, why are we, as an industry, so far off? Is it merely incompetence or perhaps fibbing to say what the suits want to hear or are there more subtle things going on, too? I think it's exactly the same problem that my little girl has... She can see the surface of a pile of cereal in her bowl and as she pours the milk, she sees the milk run off the surface and disappear, by the time she sees the milk level come up towards the surface of the cereal, it's too late.

Now, some folks might say to eat more, smaller bowls so you don't run into that milk hiding problem (which is related to the burning your mouth on the hot sauce hiding under the cool layer of cheese on your pizza :-). Others might advocate pouring some amount of milk into the bowl first and then pour the cereal to match. Anal-retentive types might do a myriad of studies (i.e., waste a lot of milk and cereal) to figure out some magical metric by which to precisely predict the proper amount of milk to use for the average cereal according to the preferences of some imaginary average person.

Me, I like an adaptive approach: Pour some cereal, pour some milk, mix it all up, and taste it. If there's too much milk, add a small amount of cereal. If there's too much cereal, pour a wee bit more milk. If it's exactly right, look for the Candid Camera folks (especially if I used up the last of the cereal and milk simultaneously :-). Gee, for some reason, this tastes the same as another process.

Bon apetit!

It's about about the language

The Furious Purpose blog entry,Mock roles, not objects mentions a paper given at the last OOPSLA on how to think about mocking in terms of roles that need to be fulfilled. Partly, the blog entry is about the benefits of TDD (Test-Driven Development). If you're not hip to TDD yet, check out Kent Beck's book Test Driven Development: By Example.

In my mind, however, there's something a bit more fundamental and interesting underlying this discussion... That is, answering the question: what's the language that is being created? In the Furious Purpose example, it's a language that is manifested as a programmatic API in Java starting with a user login story. As programmers, that fact that we are creating and manipulating languages seems about as obvious as water is to a fish but that is precisely why I think it's something worth thinking about -- we are so easily caught up in the leaves that we miss the trees, the forest, and the bulldozer that is about to run us over.

Keys to Debugging

My buddy Terence Parr just published an article on developerWorks called Learn the Essentials of Debugging. In it, he brings up a number of essential debugging facets: Reproducibility, Reduction, Deduction, Experimentation, Experience, and Tenacity. Those six are certainly important but they aren't the whole story.

The first one that I'll add (based on the classic, nagging wife story... :-) is: Ask for help! Tom did what he could to figure out the problem and then he asked for help. Unfortunately, the boys forgot to ask me (an email expert that they know) and wasted a lot of their time -- the quoted-printable issue was the first thing that came to my mind. Doh!

However, like any problem solving process, the most crucial aspect of debugging is a more subtle issue... Admit that there is a problem. Now, obviously, Tom certainly realized there was a problem because he was actually testing his code and paying attention to the results. But, look around you. Have you noticed how many "issues" are missed, denied, and otherwise ignored? Look at the various bug databases (like the Java BugParade) where real, serious issues are ignored or outright denied for years. And, heck, at least someone bothered to recognize those problems and report them. Think about how many problems are completely missed because so many developers do only lackadaisical testing (let alone documentation or design work). Even more insidious are the higher-level dysfunctions such as the head-in-the-sand denials that I pointed out in Anatomy of Insanity?.

Humane development processes attempt to address this issue in various specific ways such as pair programming, code reviews, test-driven development, etc. The fundamental technique is increasing the cyclic rate of feedback at each level so that we will get used to listening to the feedback and then act in response to what we learned.

Is it stew, yet?

Before directing you to an essay on Language Oriented Programming, I want to make one other comment about this particular pumpkin soup. We're eating it today and yet I made it Tuesday night. I learned this at a restaurant I worked in. The chef served "yesterday's soup" instead of "the soup of the day". He said that most people know that soup always tastes better the next day so that's when he serves it. This gets to John Mitchell's Slack post. It would be nice to have the time to work ahead on some project and let it sit. You could then come back to it in a couple of days and have the fresh perspective to improve it before serving.

In XP, one of the benefits of pair programming is the difference of perspectives between the current driver and the copilot. Another refreshing perspective change comes when shifting between the two roles. One of the biggest benefits of test-driven development is the forced change in perspective as you shift back and forth between test writing and code writing.

If you're not working in a pair, you can simulate this yourself by working on a task for awhile and then put it aside and come back to it later.

Obviously, the relatively short granularities on the order of minutes are good for getting some kinds of perspective, it's critical to also sprinkle in the larger granularities of days and weeks. There's nothing like a sleep cycle or three to let your brain disengage consciously but continue percolating on it unconsciously to brew some fresh thinking.

Thanksgiving, Reuse, and Slack

In Leftovers, Dan "Superman" Steinberg brings up the question of how to deal with reuse in this age of agile, lean development.

One trick is to just have a really good memory. Alas, given that most of us are human, that doesn't seem to work too well in practice. :-) Given that we're tool users and creators, we do have some options in helping us remember. A classic is a catalog of index cards -- the post-modern equivalent: a big junkyard of code (and you do, of course, keep *all* of your code in a repository, right?) and a local search engine. Another trick is creating and using an orphaned code wiki -- i.e., if the code is something interesting enough to save, add it to a wiki.

Dan asked "With coding - how do we think of reuse? Are we at a point where we start planning for our code leftovers? XP teaches us not to code for situations we don't yet need, but a customer could have a reusability story." Basically, the question is where does "reuse" fit in with the core computer science notions of necessary and sufficient? IMHO, the missing key is the notion of slack. Is there enough slack in the system to allow it to function reliably and robustly in the face of change? Most developers are exposed to this notion in practice in dealing with performance but slack is crucial in all aspects of software. To continue with Dan's food analogy, the orphaned code wiki, for example, is akin to putting leftovers in the freezer.

Big thanks to Dan for his tireless efforts in making the world a better place.

This work is licensed under a Creative Commons License.