Skip to main content

Introducing java.net issue tracker stats

Posted by kohsuke on March 2, 2008 at 8:41 PM PST

For various reasons, I haven't touched my java.net tasks project that much lately, but I'm thinking about putting a bit more effort in doing more java.net automation, especially around issue tracker. Toward that, I wrote a tool to generate graphs out of the issue tracker — I'm calling it "issue tracker stats".

The tool is probably more useful for the management to get insights into projects, but just looking at them is rather fun and revealing.

For example, the following picture shows the number of resolved issues (green) and the number of open issues (red) over time in the glassfish project. What's truly noteworthy here is that the team managed to actually reduce the number of issues during early last year. This is rather unusual, because it's normaly for a project to receive certain amount of long-term RFEs/bugs that cannot be fixed right away.

The following picture looks at the same information in slightly different way. Instead of plotting the total number of issues in Y-axis, this graph shows "incoming" issues (newly filed and reopened issues, red) and "outgoing" issues (resolved issues, green) per month, over time.

You can see the correlation between this graph and the graph above and infer a lot. For example, the dent in the number of open issues is now visible as the region where the green bar goes above red bars, but now you see that that time period is preceeded by a surge of bug reports (Aug,2006-Jan,2007), so perhaps what happened was that the team went off to do feature developments, and then decided to come back to the bugs later.

A quick check of the Aquarium reveals that GlassFish v2 released September 2007, and if you look at the graph with that knowlege, you can now see that the team has been in the "bug scrub" mode for almost 3 quarters. The number of incoming issues didn't jump after the release, so that probably suggests that the quality of the code was good (or maybe people aren't trying newly released v2.)

Yet another way to look at the information is to see what happens to issues after they are reported, which is what the next graph is all about.

The graph is again taken from the glassfish project, and it shows what happened to issues X days after the report. For example, the yellow region hits 50% at around 15 days and green hits 50% around 45 days, so you can tell that by 2 weeks, 50% of all the reported issues get started, and within 6 weeks about half of the issues are resolved.

You can also see that more than 15% of the issues aren't even confirmed after 100 days — that is, many issues aren't even getting evaluated.

Comparing these graphs between projects is also very interesting. Here is the same created/resolved graph from my Hudson project:

Now you can see that resolved issues are always outnumbered significantly by incoming bugs/RFEs. This clearly shows that the project is getting more feedback than it can keep up with, so it probably need more developers.

The following graph is the expected longevity graph from Hudson. Contrast that with glassfish, and you can tell a lot. First, initial declines are a lot steeper (note that the scale of X-axis is different) — in fact 50% of the issues get attended within 3 days, and most of the issues, if they ever get fixed, get fixed within 2 weeks. So the turn-around time is lot faster compared To glassfish, although more than one third of the total issues remain in the issue tracker.

Or if you look at the following graph, which is taken from Metro, you can see the declining incoming issues, which is hopefully a sign of increased stability in the code. You can also see that the bug fixes are spikey, probably indicating the release cycles of the development team.

The tool itself is available on java.net, so try it on your favorite projects. If you have more ideas about the kind of statistics that are useful, let me know (or better yet, join the project as a developer.)

Related Topics >>

Comments

Eduardo, do you read my mind? In fact t hat's what I was working on last night, so stay tuned...

Some automation via Hudson would see useful - eduard/o

very good! I will definitive try it out.

>Never have seen that on any java.net project, seems to be highly secret ;-)
I bet his stats project waits for approval. AFAIK all content is just accessible for project owners until it is approved.

Hmm ... would love to give it a try, but aren't allowed to access issue-tracker-stats, the error:

Error Your account does not have the "Project Page - View" permission needed for you to access the page you requested in the issue-tracker-stats project (view your permissions). Either ask the project administrator for more permission, or log in using a different account.

Never have seen that on any java.net project, seems to be highly secret ;-)

Jeanette

Ah, yes, the issue was that the project needed to be approved before becoming visible to others. I guess I jumped the gun.

It should be now visible to anyone, though.

cool - run over SwingX immediately :-) And intend to do it periodically, every quarter/half year of so opened a page in the swinglabs wiki which will reference the results.

Could we integrate that easily enough (for Jan, not me ;-) to our Hudson tasks?

Jeanette