 |
The New Golden Hammer
Posted by tball on October 19, 2004 at 11:48 AM | Comments (7)
I think all Java developers have at one time or another fallen in love with a new technology and wanted to use it to solve all problems; after all, when you have a "golden hammer", every problem starts looking like a nail. Remember when Java Reflection was all the rage when it was first introduced? All of Java's perceived ills could be solved with it, provided one ignored how slow and unreadable the solution using it was. Now the "first love glow" has subsided, and reflection is now an important but specialized technology in our toolkits.
With the release of Tiger, annotations are rightly getting developers excited, but its companion tool, apt, is starting to look a little too golden in my view. Last week, it was suggested that apt should replace javac, and while I applaud that sort of "out of the box" thinking, others are taking a cool thought experiment too literally. Users have already complained on the NetBeans forums about how its upcoming release doesn't support this "feature", even though the apt Mirror API doesn't support code types that a compiler needs.
I think what bothers me about wild apt scenerios is that folks are forgetting (or ignoring) the problem annotations are trying to address: that much of the specification for a class is done outside of its source code in other files, or is done in repetitive boilerplate code that has little to do with the class's problem domain. Information about a class is called metadata, and annotations allow developers to define metadata without external files (other than new annotation type declarations), special naming patterns or other error-prone hacks.
Java developers have been creating ad-hoc solutions to this problem for years. If you look at the Swing sources, for example, you'll find undocumented "@beaninfo" Javadoc tags, which a custom doclet in its build uses to create its BeanInfo classes (XDoclet is an excellent tool that takes the grunt work out of such metadata file-generation doclets). Another tool that adds pre and post-condition checking via custom doclet tags is JContract.
The problem with these approaches is that, like all comments, metadata stored in comments can drift away from the source if not carefully tracked by its developers. It's also fragile, in that this metadata isn't checked for typos; for example, try misspelling "@deprecated" in a doc-comment and notice how the compiler just ignores it as comment text, instead of either recognizing it or reporting it as an error.
The Java 1.5 annotation support doesn't fix all metadata problems, but it helps in several ways. First, annotations are true Java types whose definition and usage are verified by the compiler. This makes it much harder to break metadata with a typo, but more importantly it means that if you enhance the definition of an annotation type and forget to update a class that uses it, the compiler will catch that error. I'm very partial to letting the compiler do as much project validation as possible (followed by my unit test suite). It also allows metadata to be discoverable in a standard way in classfiles and at runtime, so that tools like deployment tools or introspectors don't have to use external files and classes.
What makes apt and annotations so exciting to me is that anyone can write an annotation processor as easily as an XDoclet module, writing metadata using a verifiable Java language facility instead of comment text. So instead of trying to see how apt can replace existing tools, look at all the metadata your project has and see how it can be better handled using annotations. If your classes take more than one file to define (such as a factory or "info" class), has boilerplate code in it or doc-comment tags that aren't documentation-specific, consider defining it using annotations. If you need a compiler, stick with javac or jikes.
Bookmark blog post:  del.icio.us  Digg  DZone  Furl  Reddit
Comments
Comments are listed in date ascending order (oldest first) | Post Comment
-
Leaping to my own defense
Have you ever used a screwdriver to open a can of paint, or your car key to extract a stone or drawing pin wedged in the sole of your shoe, or a coin to open the battery compartment of a whatever, or a paper clip to open a powered off CD drive, or used a pair of pantyhose to either tie a rose bush to its stake, or filter paint?
>I think what bothers me about wild apt scenerios is that folks
> are forgetting (or ignoring) the problem annotations are trying
> to address
Please don't be bothered, resourceful people have acted like this for as long as they have used tools.
How I use a tool is up to me. If I can get benefit from it (and cause no harm) I am quite entitled to use it for things the original designer had not considered.
I am neither ignoring nor forgetting the problem annotations are trying to address. However, I see the problem domain where apt might be appropriate as much larger than that. I cannot be sure whether this is larger than what the apt designers (as opposed to annotation designers) were intending, or not, nor whether what I can do was intended or is just a side effect of the design, but actually I don't even care.
Certainly there are hints in the apt documentation that suggests it has uses beyond annotations
"While intended for annotation processing, apt can be used for other reflective programming tasks too"
From Getting Started with
the Annotation Processing Tool (apt)
If I can detect runtime errors at compile time, by using apt as my compiler instead of javac or jikes, then I will. If others fail to benefit from this new checking functionality by not changing to apt as a compiler, then I believe there is some merit in proposing that the old compiler be deprecated and the new extensible compiler that supports this becomes the replacement.
Note that I am not saying it must happen, I am merely making the suggestion, so that others making the decisions are at least aware of the possibility.
Using annotations and apt as a way of extending the compiler is doing nothing more heinous than the XDoclet and JContract tools you talk so positively about. All these are taking a standard language feature and tool, and putting them to new uses. Whether it is doc comments syntax and javadoc tool, or annotations syntax and apt, is irrelevant. I fail to understand how you can be so positive about Xdoclet and JContract, and so negative about using apt as an extensible compiler.
Posted by: brucechapman on October 20, 2004 at 06:00 PM
-
Switch to de-caf, Bruce, as I wasn't attacking you! :-) I wrote that it was a cool idea, and I wish you good luck investigating it to your heart's content. As someone who has worked on javac for a couple of years, I suspect you'll hit some speed bumps along the way, but don't let that stop you from trying.
What angered me was a NetBeans forum message complaining about its lack of support for apt as a replacement compiler -- less than two weeks after you first proposed the idea publicly, people are already whining! You and your idea were never the problem.
But I'm not a big fan of XDoclet or JContract or any other tool that adds metadata to Javadoc tags, even though the Swing BeanInfo generator I mentioned was my idea those many years ago. As I pointed out, anything in comments risks drifting away from the actual source code. Annotations are therefore a much better means to accomplish these sorts of tasks. I have always been interested in tools that support metadata, but up until 1.5 was never happy with the support burden they placed on developers.
So have fun, and if you come up with a better compiler, great!
Posted by: tball on October 20, 2004 at 06:44 PM
-
< Switch to de-caf, Bruce, as I wasn't attacking you! :-)
Point taken, except I drink tea. :-)
Actually, I think what I wrote probably came across a bit caustic which is not what I had intended. Sorry about that. I can't put a finger on it anywhere, but I suspect we are both motivated the same way, and are pretty much on the same wavelength.
What I can't agree with is the idea that came through your blog (possibly unintentionally), that we should only use these tools for what they were intended to be used for.
Can you post a link to the netbeans forum discussion, or add some detail on what you meant by
even though the apt Mirror API doesn't support code types that a compiler needs?
Specifically what "code types" are not supported by the Mirror API?
If netbeans can't use apt as a compiler, then I think that is something that needs to be addressed. Sure if you are generating deployment descriptors and such like which aren't accessed till deployment, it doesn't matter, but there are valid use cases for annotations/apt where you must have the "generated source" (I know thats a contradiction - I'm wrestling with terminology as well as everything else) available at compile time, or the manual source just won't compile. Or worse, it tries to compile against previously generated but out of date source.
What I find fascinating is the evolutionary process which just keeps repeating.
Each new feature allows us to do something completely new and unintended, albeit rather primitively. But we do it anyway, we learn when it is good to do it, and when not, and by having some crude concrete examples to think about and talk about, we are then able to start abstracting the concepts.
Eventually we gain enough experience, understanding and courage to directly support this new thing, rather than hack it, and this becomes the next new feature.
Without javadoc, Xdoclet probably would never have happened.
Without Xdoclet and friends, I doubt the expert group would have had the experience to do such a good job designing annotations. In fact, we wouldn't have even had any idea why we would want annotations, or even what they were.
Now that we have annotations, they will be great for the stuff Xdoclet used to do, but ,as you say, can now be done much better (safer, cleaner etc), with annotations.
But that is only half the story, it will also provide another layer on which to hack what may or may not become the next fancy feature. The feature which will do what our hacks do, but far better than we can now. At the moment we don't know what those features might be, but if nobody hacks, if nobody asks "what else can I use this for", they won't ever happen.
Posted by: brucechapman on October 20, 2004 at 08:15 PM
-
Yes, I suspect we are in "violent agreement", where two engineers fiercely argue the same side of an issue. But de-caf tea is still available, BTW, but it sucks. :-)
The issue with a golden hammer is that a tool starts getting used for all tasks, not just unconvential ones. I like unconvential use, if it gets the job better. Remember I was using javadoc for code generation back in '98 with Swing's beaninfo tags, so please don't put me in the stodgy camp.
NetBeans 4.0 can support apt as a javac replacement, if you are willing to either use the Java task to invoke apt as a command-line tool, or subclass Ant's Javac task and add support for its commands. Either option is easy, but apparently not sufficient for this developer.
The missing Mirror API I mentioned are any way to get into the body of a method, to get at its code. In its overview, it explicitly states: "Constructs below the method level, such as individual statements and expressions, are not represented." javac has forty-three tree node types for dealing with the constructs below the method level, and visitor classes to reach them easily. I'd like to see that level of support in any replacement for javac.
Posted by: tball on October 21, 2004 at 08:30 AM
-
Then your golden hammer claim is unsubstantiated, because no-one is suggesting they want to use apt for all tasks.
About the mirror API, sure more stuff could be done with apt if the method body were accessible, but that is completely irrelevant to the discussion about whether apt can or should be used as a compiler.
I am presuming netbeans uses javac, rather than its own compiler, even tho' it probably builds its own AST.
The limitations of the mirror API are relevant if you want to use the mirror API as netbeans' internal representation AST for netbeans tools, but that is not what was being requested.
From my reading the user was just saying that he would like to tell netbeans, that when it does a build, it should use apt, and these are the options (maybe factories and factorypaths, probably a generated source (-s) directory). For sure he probably used a little more passion in his post than either of our cultures find tolerable, but hey, thats what you get with a "fr" domain name.
There appears to be some blurring in your remarks between compiling (with apt or javac), writing a compiler, and writing a parser for an IDE. And where the Mirror API fits in to all these.
The limitations of the Mirror API do not prevent you compiling with apt, but they do prevent you using the mirror API to write a compiler or parser.
I am not writing a compiler, I am merely adding functionality to one (apt), using the mirror API.
The only way I can make sense of some of your remarks is if you were imagining a system where an annotation processor used the mirror API to generate bytecode for the corresponding source code. (that is, the processor did the compiling but used apt to do the parsing).
These are the remarks in particular:
Users have already complained on the NetBeans forums about how its upcoming release doesn't support this "feature", even though the apt Mirror API doesn't support code types that a compiler needs
If you need a compiler, stick with javac or jikes.
So have fun, and if you come up with a better compiler, great!
javac has forty-three tree node types for dealing with the constructs below the method level, and visitor classes to reach them easily. I'd like to see that level of support in any replacement for javac.
If that is the case, I can't see where you got that idea from, and if it was from something I said, then I am really sorry that I threw you so far off course. If you tell me what it was, I'll try to do better next time.
Posted by: brucechapman on October 21, 2004 at 03:33 PM
-
I've been using javac as a tool base much longer than I've been working on NetBeans, when I was doing research on Jackpot. So when someone mentions tools and compiler in the same sentence, I'm inclined to think of semantic-based tools which derive their power from a very deep, correct understanding of source code. And for that, you need a deep representation which the Mirror API doesn't have (yet). So I guess my biases were showing.
BTW, you weren't the only developer interested in apt's potential, which is why I am perplexed that you took my blog entry so personally. There is a lot of traffic regarding different potential apt uses, and some of them are very golden indeed.
Posted by: tball on October 21, 2004 at 05:08 PM
-
BTW, you weren't the only developer interested in apt's potential, which is why I am perplexed that you took my blog entry so personally. There is a lot of traffic regarding different potential apt uses, and some of them are very golden indeed
Mine was the only one you linked to (even if you did say nice things about it)
I didn't agree with what you said in the very last sentence.
My opening Leaping to my own defense statement was meant with at least a little jest, and was intended as a disclosure about my authorship of the "deprecate javac" post, so that others could read my comments bearing that in mind.
So all in all it was more about understanding where you were coming from, and correcting a few assertions I believe you got wrong, than taking it personally. The subsequent dialog has pretty much cleared all those up. Thanks.
Posted by: brucechapman on October 21, 2004 at 05:47 PM
|
java.net RSS Feeds
