The Source for Java Technology Collaboration
User: Password:



Konstantin I. Boudnik's Blog

March 2006 Archives


New Java VM blog is coming...

Posted by cos on March 31, 2006 at 10:04 AM | Permalink | Comments (3)

Hi there

Again a message mostly for those, who is speaking or at least reading Russian. Our folks from Saint-Petersburg I was writing about this great city before SQE team are starting their own blog, designated for the Russia-speaking development crowd.

The blog, I believe, will be a real first hand's value-add for existing Java VM resources, about modern VM techniques, tools, security features, co-existence with .NET, etc.

There isn't much right now, but the folks will carry on fast, I think.

You can find this new information resource at at the official Sun blogging site

So, if you have colleagues or friends, who are suffering without such information, written in Russian - please pass by my message or simply the URL above. Let's wish them good luck and God speed!

--
Thanks
Cos

Java APIs comparison

Posted by cos on March 28, 2006 at 09:57 PM | Permalink | Comments (9)

Hello folks

We recently thought about an interesting approach of finding a difference between APIs (public methods) of two versions of a Java software. I was doing it for JDK1.5 (Tiger) and JDK1.6 (Mustang b76).

Sometime you might want to take a quick look and find out what exactly has been changed in your (or somebody's else code) between two versions. Why? Well, if you want to focus your test development on the new API only it might be a very sound idea. I'd spoke about some of other methods to help you to achieve better software quality
Well, there are some methods to do so. E.g. you can use "Since" tags from a Javadoc Well, good if you have any...

Or you can store your product API's snapshot like RefactorIt does. Boy, what if you didn't create this snapshot in a first place and now you badly need one? "Sorry partner - no can do for ya..."

Anyways, the idea was on the surface as usual:
In one way or another build a list of all public (protected/private/static/etc.) methods in your software. You can use grep or javap along with grep for these purposes. Don't forget to do this for both versions of your application - it's about the difference, right? :-)

Now, you have these API lists, you just need to compare them. And again it's up to you: you can use Unix diff command, Emacs ediff, or vimdiff. I'd wrote that self-explanatory Perl script (feel free to modify it, criticize it, or merely throw it away). Fix &open_file()'s loop if your API lists weren't created in the form of "filename:method signature" per line...

#!/usr/bin/perl

if ($#ARGV != 1) {
    print "Please supply two lists of APIs you want to compare\n";
    exit 1;
}

my %diff = ();
my $left = &open_file ($ARGV[0]);  #Presumably, the earlier version
my $right = &open_file ($ARGV[1]); #Presumably, the latest version

foreach $file (keys %$right) {
    if (!exists $left->{$file}) {
	$diff{$file} =  \@{ $right->{$file} } ;
    } else {
	@l_stuff = @{ $left->{$file} };
	@r_stuff = @{ $right->{$file} };
	foreach $r ( @r_stuff )  {
	    if (&is_there($r, @l_stuff) != 1) {
		push @{ $diff{$file}}, $r;
	    }
	}
    }
}

foreach $d (keys %diff) {
    @extract = @{ $diff{$d} };
    print "$d\n   @extract\n";
}

#=========== Some subs
sub is_there () {
    my ($search, @list) = @_;

    for (@list) {
	return 1 if $_ eq $search;
    }
    return 0;
} 

sub open_file () {
    local ($name) = @_;
    open FILE, $name or
        die "Can't open specified file!$name\n";
    my %ret = ();
    
    while () {
        ($key, $value) = split(':', $_);
        push @{ $ret{$key} }, $value;
    }
    return \%ret;
}

Of course, there's always an expert opinion. Or you can ask an author of an application, or eye-roll it yourself. But isn't that much simpler to load your computer with the stuff it can do better then a human being?

So, I'll let you to carry on from here...

CU,
Cos

Java. Quality. Metrics (part 6)

Posted by cos on March 15, 2006 at 05:20 PM | Permalink | Comments (0)

Hi there.

In this short article I'll try to summarize what I was discussing for the last couple of months.

So, let's briefly list key factors that are likely to affect our judgment of software quality. - our code quality expectations (good enough quality, remember?) - coverage isn't everything - code complexity and a frequency of the changes - number of bugs filed against source code modules/files - testing methodologies

Alas, the last one doesn't sound like a beast, it might reduce the effectiveness of defects discovery rate a lot. Obviously, it is a choice of approaches of test failures analysis. The bulky one with a weak algorithm of false positives detection pisses off engineers and they begin ignoring most possibly important warnings and notifications.

Anyways, I want to talk about a combination of the first three bullets above.

About a year ago, a few of Sun's fellas were chatting about simpler ways of delivering a better code. Static analysis and variety of testing approaches were among the things on the table. At some point, the bright idea of mixing both of those and adding some other flavors had appeared. Afterwards, we came up with what was called Buggy Spots Prediction (BSP).

The idea itself is as follows:
  • we're creating a static call graph (CFG) for any given source code, using a commercial or home-grown tools
  • having this, we can calculate a few things about this graph, e.g. the frequency of calls to any particular method; the frequency of calls from a method to other methods with-in the code; basic-block based complexity of a code, etc. (currently, we calculate about five or seven of them, i.e. coverage per function, basic block per function, et cetera)
  • when executing tests against the instrumented build, we can prepare a code coverage metric for it
  • combine these two lists of modules - from CFG and from code coverage runs - by module names
  • sort the resulting list ascending by in-call frequency and descending by coverage scores
  • let's assume that most frequently called functions are, perhaps, most important from the quality standpoint. Well, their code is called more often, so any problems will immediately affect a top-level or at least quite important functionality.
  • if such methods are having low coverage numbers and high complexity or high number of reported bugs, then it might be a good indication that the code has to be targeted by quality engineers and/or developers.
So, all that is giving you a way of quickly selecting possibly buggy spots, e.g. the pieces of the source code which are likely to become a root of defects found by your customers. Why? Well, simply because of the fact that the coverage is low in these areas and existing tests aren't guaranteed an acceptable level of quality.

As any heuristic approach, this one might produce incorrect results. However, our preliminary predictions are quite coherent to the fact that most of externally reported defects were found in the poorly covered but frequently called methods.

In organization with limited QE resource, a manager might want to firstly address such hot spots. This will help achieve a good-enough quality level and then concentrate on less important issues.

Yet another benefit is that the technique is a language independent. Once you'll build a universal presentation for CFG and code coverage information, you can use the same engine to measure Java, C++, and programs written in other languages.

And, of course, our methodology doesn't replace a human knowledge of the importance of product features. It helps engineers see a valuable projection of static-to-runtime boundaries and helps focus on some aspects of that complex matter.

And I just want to remind to you about Project Mustang (Java6) Regressions Challenge. Please check here for more details.

CU,
Cos

Hidden problems of the blogging

Posted by cos on March 06, 2006 at 10:34 AM | Permalink | Comments (0)

Hi there.

It's rather short message about a potential problem you might face as well.

Recently, I've been told that few of my blogs are having incorrect links to other resources. Being very careful about posting any materials, I was quite surprised and went to http://weblogs.java.net/blog/cos/ to check it out.

In a short time I realized that a couple of the articles were typed using StarOffce's document writer and were saved in OpenDocument Test (ODT) format. As the result, all quotas - " - were replaced by fancy typographical quota symbols. When they became a part of an URL they were encoded into something like
%E2%80%9C
for left " and
%E2%80%9D
for the right one

That is well know fact, but is a good illustration of what kind of weird bugs you might discover sometimes during your quality cycles.

Cheers,
Cos



Powered by
Movable Type 3.01D
 Feed java.net RSS Feeds