Skip to main content

Teamware, Mercurial, and SCCS revs that go bump in the night

Posted by kellyohair on September 27, 2006 at 9:09 AM PDT


As seen in Mark's recent blog on the JDK and
Source Code Management (SCM) solutions at

http://blogs.sun.com/mr/entry/openjdk_scm
,
you can see that we are finally looking at converting to a new SCM.

Inside Sun the historic and
most common SCM used has been Teamware.
The Sun Workshop
product had delivered Teamware with it's compilers and tools
for many releases, but the newer Sun Studio product dropped
Teamware from it's feature list many years ago.
Of course this didn't stop the developers from continuing
to use it.
The best high-level description of Teamware I could find is at

http://docs.sun.com/source/806-3573/intro.html
.

Besides all the basic SCM features you'd want,
Teamware provided two extremely valuable features in my view.
The primary feature was
the ability to have any number of workspaces (repositories)
with a parent/child relationship, there is no single repository.
The second feature was it's extremely powerful
merging tool called filemerge.
Luckily filemerge can work with
any SCM, and over the years many source merging tools have come
into the picture, so as much as I loved filemerge, it's not that
critical a SCM feature anymore.
But that first feature of having essentially a tree of repositories
allowed us to do a sort of 'divide & conquer' or
distributed approach to development.
Any group of developers could spin off a team workspace and a child
workspace per team member, completely independent from the master
workspace.
This distributed development model allowed teams to work in complete
isolation, yet have all the SCM tools available to them at
any number of levels.
Well, bottom line is, we just can't give this up very easily.

At the core of Teamware was SCCS which managed source file changes
on a one by one basis.
A Teamware workspace is essentially a set
of SCCS controlled source files, with a bit of frosting.
If you really want the nitty gritty details, take a look at

http://docs.sun.com/source/806-3573/underhood.html

It's these SCCS files that contain the various revisions of
source files, along with comments and details from the developer
when a revision of a source file was created.

As Mark stated in his blog, we are strongly leaning toward Mercurial,
but we need to study the impact of the conversion.
Martin Englund has been looking into the

SCCS migration into Mecurial

so we have a feeling of the size and performance differences.
Just copying files into a new SCM isn't the issue here, it's the
conversion of all the file revisions and the information
that goes along with each revision.
As an example, the Hotspot workspace alone contains roughly
3,300 files, but has a total of 140,000 individual file revisions.
And Hotspot is one of the younger workspaces in our JDK workspace
corral.
We know (thanks to Martin)
that Mercurial can handle the 3,300 files with amazing
speed, and Martin even tried 40,000 files. It was very fast.
Now we need to see what happens when it gets all the revisions too.

Teamware did not provide a very good 'change set' model, the
workspaces had individual history of putbacks or commits,
but the history of these actions didn't propagate up the tree
of repositories.
So we lack any good Teamware history data to help us group the
individual SCCS file revisions together into logical 'change sets'.
So for the most part we are left with the individual SCCS revision
data for thousands of files.
So we are brainstorming how we can group individual file changes
together into change sets with just the:

  • SCCS comments
    (in theory the files of a change set might have the same comment)
  • possible bug ids (7 digit numbers) in the comments
    (in theory the files of a change set might have the same bug id)
  • user id's
    (in theory a change set would have the same user id)
  • times with no time zone :^(
    (maybe in theory, the files of a change set would have been changed
    within 24 hours of each other?)

Why not just do a blind 'one file revision' to 'one change set'?
I guess it's part of being an engineer, we can do better than that.
:^)
And it looks like a fun graph project too, humm, maybe I can get some
use out of that Mathematics degree I have?

Of course, if we figure this out, and I don't get fired for
doing a horrible job of it (just kidding), this SCCS revision
conversion work could map to any of SCM's with a change set
model, but we will be focusing on Mercurial for the time being.

The Open Solaris team did a fairly detailed study on SCMs which is
available at

http://opensolaris.org/os/community/tools/scm/
.

Mercurial is an open source project and details about Mercurial
can be found at

http://www.selenic.com/mercurial/wiki/index.cgi
.
Martin Englund has also been nice enough to provide a
Teamware to Mercurial mapping for the existing Teamware
users at

http://blogs.sun.com/martin/entry/mercurial_for_teamware_users
.

No transition will be easy, and we know that some of you might be
upset that the choice wasn't SubVersion, but for what we need to
do in the JDK, with our JDK developers all over the world,
our answer wasn't SubVersion.


Related Topics >>