David Walend's Blog

Community Archives

Better JavaDoc on http://java.net

The Problems with using CVS for JavaDoc

I have two problems with using CVS to store JavaDoc. The first is philosophical -- CVS is a source code control system, and JavaDoc .html files are a derived product, not source code. If you can check out the source code, you can build the JavaDoc yourself. If you change the JavaDoc, it doesn't matter; next time you rebuild it, those changes are lost. The second problem is logistical -- I either have to remove all the JavaDoc from the last release and add all the new JavaDoc from the latest release. Or I have to put each release under a new directory; that saves me removing it, but means that everyone's links point to an old copy and grows the www subdirectory over time. Either way, I have to add several layers deep of JavaDoc directories. And I get a 15 MB cvs change log in email every time.

Sourceforge's Approach

Sourceforge gives each user a unix account and each project a public web directory. Back when I kept projects on Sourceforge, I ftped a tar ball of .html files for the web pages, including the JavaDoc. It worked, but was fairly primitive. Plus it relied on letting the users log in remotely to sourceforge's servers. That's probably more than Sun, O'reilly and Collabnet are willing to pay for.

I've thought of three alternatives, but don't know which would be the best.

Subversion

Collabnet, one of java.net's sponsors, paid to create Subversion, a replacement for CVS. I've used it at home for a while, and on a few projects at work. I'd rather use Subversion than CVS in any case; Subversion treats each commit as a transaction, linking all the changes together. Using Subversion to store source code and web pages would still not address my philosophical concerns. However, Subversion's ability to easily remove and add whole directory structures would be a huge improvement over CVS. I could even get rid of the top level v1, v2, v3, etc. directories.

.war Files

In the previous blog, straun suggested a separate repository of .war files for web sites. Using .war files would let us maintain project web sites the way Java developers are used to. The approach addresses both of my concerns. When I'm ready to release a new version of the web site, I build a .war file and post it in the appropriate place, instead of checking in the files as source code. And I handle the site as a single .war file instead of single files in directories. How close is http://java.net from being able to support that? What other pitfalls are involved? Would the administrators have to do something special to keep developers from putting executable code in the .war files?

Shared JavaDoc Database

A more compelling solution: Join with JDocs, a shared cross-linked JavaDoc database, or to set up something similar on java.net hardware. The JavaDocs would be completely separate from source code. And over time, we would grow a large, cross-linked set of API docs that would help us use each others' work. The JDoc site hasn't added new material since September 7th. I don't know what would be involved for the java.net administrators to set up such a system, or the java.net sponsors to link up with JDoc.

Those are the options I've thought of. Which of these should we ask for? Is there some other option we should consider?

Thanks,

Dave

Moving JDigraph

Spring Cleaning

Both sourceforge and java.net use CVS for source code control. Changing directory structures in CVS is a pain; in CVS, files really can't be moved around or renamed without loosing their history. CVS commands for directories feel like an afterthought. As a result, my directory structures in CVS projects tend to calcify into whatever I try first. So I took the opportunity to reorganize the source code tree, using ideas I've learned over the last few years.

Ant's Import Task

Changing the directory structure meant reworking the build.xml files, another chore I'd put off for half a decade. I considered shifting to Maven or Jython for the builds, but ultimately decided to stay with Ant. Maven gives a nice canned build, but it doesn't match my environments well (java.net, sourceforge, the day job). Jython would give a more powerful language, but leave behind entry level developers who can pick up Ant in a day. Ant 1.6's new import task really provides what I want.

For years, I've had almost identical build.xml files copied through many projects and subprojects. Ant lacked a nice way to pull functionality from one build file to another. (I know a few XML tricks, but those are more challenging than Jython.) JDigraph demonstrates that import was the main thing I was missing. Here's what a subproject's build file looks like now:

<project name="collection" default="test" basedir=".">

 <import file="../common/common-build.xml" />

<!-- set global properties for this build -->
 <target name="set-subproject" >  
  <property name="subproject" value="collection"/>
 </target>

</project>

Into java.net

After I rearranged all the code, checking it in to dev.java.net was easy. CVS on java.net is a little more trusting than on sourceforge; I only had to provide my password at the beginning of a session, instead of during every operation.

JavaDoc on java.net

The only real obstacle I hit was getting JavaDoc (and the rest of the web site) up on java.net. After a few rounds of email with the java.net support crew, I took their advice and put the whole web site into source code control, under the www/ top-level directory. (java.net's support is great. Thanks, Helen.) I can hear people I've worked with gasping in amazement; one of the big tenets I've held over the years was "never check in anything to source code control that comes from the build." My big concern is mechanical. If I have to check in all of the javadoc every time I do a release, then I have to somehow disconnect the old javadoc. CVS is not so good at removing whole directories. Tracking down all the obsolete javadoc files is even worse. So I created a directory under www/ called alpha-0-7-2/ (the name of the current version), and checked in all the source code there. Next release, I'll create a new directory for the javadoc; the build already puts in the correct link on the project's home page.

The CVS mailing list for JDigraph rewarded my cleverness with a fifteen megabyte CVS log.

The next problem was the javadoc didn't show in java.net's frames. Another email to java.net. Helen said to add "nonav" to the beginning of my links. ("nonav." "No navigation." I didn't get it for about two hours.) It worked like a charm.

Finding a Home

java.net is focused on about two dozen communities. That wasn't the case in June 2003 when I first reserved a spot for jdigraph. This summer when I finally moved JDigraph in, it was an orphan. At JavaOne, I spoke to a few people involved with edu-research at JECL, which seemed like a good fit. A few emails to Daniel Brookshier and Robert Stephenson closed the loop. I get the impression that a body of working, interesting, relevant code would get any project in the community (and an interesting idea would make the incubator).

What Could Be Better?

Helen knows I'm going to ask for Subversion. (She's not above encouraging me.) Wrangling the javadoc in CVS is labor. Switching from CVS to Subversion would answer most of the problem. I'd still have javadoc -- a product -- in source code control, but Subversion would make the job of swapping in a new version easy. Collabnet helps run java.net, and paid to have Subversion written. It seems like a natural move. The word from the support crew at java.net is that we'll have it someday, but it might be a while.

Reviewing the Java Community Process v. 2.6 (JSR-215)

My experience with commenting on JSRs lines up with that of many other people. We squeezed a review of a public draft into the thirty day window, wrote up and sent in our comments, then... nothing. We received no feedback, and saw no changes in final draft. Reviewing a JSR is a significant investment. Reading a specification may take several days of concentrated effort. Writing up useful comments takes several hours, especially if those comments include cross references and code examples. For an example of comments on a JSR, see Bob McWirter's comments on JSR-94. (I'm using a few references to Bob's web pages. His projects are always fun to watch take shape.)

Not hearing anything back, and not seeing any change from public draft to final has created a perception that the JCP allows industry powers to dictate the standards for the rest of us. The expert group forms, presents a public draft, and then the process is over. That perception may not be fair, but it is prevalent. Several members of the audience expressed it at a special meeting of the NEJug. See Bob McWirter's experience with JSR-94.

JSR-215's public draft for JCP 2.6 encourages more transparency. A new paragraph on page 4 requires a new JSR proposal to include a transparency plan so the public will know what to expect. Another paragraph on page 9 encourages a spec lead to be more open. Pages 10 and 11 promise access for the public to an early draft, no longer only accessible to JCP members. And "Members have a right to receive a response to comments."

According to Doug Lea, JSR-166 is an example of how a more open process should work. The mailing list this project operates has been fantastic. All suggestions are weighed, deliberated and sometimes the expert group changes the spec. If more JSRs are run like this one, JCP 2.6 should fix the problem.

I think the new phrasing in JCP 2.6 goes as far as it can. I doubt Sun's lawyers will let anyone talk to an expert group without signing the JSPA. I don't think the expert groups will be overwhelmed with comments simply because making comments takes time, expertise and effort.

Does JSR-215 go far enough? I hope so. Next time I comment on a JSR, I'll know what to expect back.

This work is licensed under a Creative Commons License.