Skip to main content

Mustang regression contest is now over

Posted by robogeek on April 1, 2006 at 8:33 PM PST

I probably should have posted this last night, but it was pretty late. As scheduled in the contest rules, at midnight, last night, March 31, 2006, we closed down the regressions contest. Which meant editing the HTML for the contest site and disconnecting the bug submission page.

What's left is to process the final submissions, get the winner selection committee formed, and select the winners. We expect to have that finished by mid-April.

This has been an amazing experience. We started this contest without a clue what would come of it. Thankfully the management trusted that something useful would result.

Would we get 100 submissions, 1000, or 10? The number of submissions would directly determine the number of people required to screen them. But how many would that be? There was no guideline we could look at, because this kind of thing (the contest) is rarely done. The best estimate was to look back in history and we noticed that during Tiger Beta there were some number of regressions submitted. Hence, we could expect approximately that many to come in for the contest period, which roughly coincided with the beginning of Mustang Beta. We're going to have to dig into the numbers, but it appears we exceeded the number of regressions submitted during Tiger Beta.

We didn't know, would you laugh at us or otherwise tear us apart for launching this contest. Yet, you accepted the contest and gave us some very good bugs. The only serious quibble is the unfortunate situation with the set of countries from which we could accept submissions. The root cause for that is, as I explained earlier, the widely varying laws for running contests. We worked with some lawyers to set up contest rules that could work in as many countries as possible, but unfortunately some countries put up too many roadblocks.

I am very happy with this contest. This was my first time leading a large project anywhere, and it happened so smoothly. The contest team numbers around 20 people who seemed totally bought into what a neat idea it was to have this contest. They stepped up to the plate over and over to do what needed to be done. I am so grateful to them for their help.

And, I hope we managed to demonstrate to you our commitment to Java Quality. Of course we have to follow through with the promise, to fix or address all the bugs you reported. In truth the contest made no extra promise, because that's how regression bugs are treated anyway.

Now that the contest is over please continue telling us about the problems you see. We cannot fix these regressions unless we know about them. We'll find some ourselves, because of the extensive testing we do. But there's some that only you can find.