 |
See Java code the way javac sees it
Posted by timboudreau on February 28, 2008 at 03:50 PM | Comments (22)
I wrote a small Swing app to browse javac's abstract syntax trees (ASTs) of source code. If you've ever wanted to write a tool that analyzes Java source code, it is a big help for understanding the Javac tree API.
Last spring I set about learning javac's Tree API (JSR 199) - NetBeans uses it internally as its parser for Java source code. It has a bit of a learning curve, and contains lots of this-tree and that-tree interfaces for different types of structures in Java source code. So to help me develop an intuitive understanding of what all of these classes map to, I wrote a tool - a standalone Swing app that lets you open a Java source file and then browse its AST.
The tree on the left lets you drill through the syntax tree itself. The middle pane shows you the object that is selected, and a tree of all of its fields and methods via introspection, which you can drill as deeply as you want through. The right pane is sensitive to selection in whichever of the other two panes has focus. The top half shows you the list of all types the selected object implements (and boldfaces those that are part of the official API), and the bottom half shows you the value of toString() for the selected object (if it's part of the syntax tree, it will be the source code text).
I found it very useful, and still use it to figure out just how to hunt down a particular thing I want to find in source code. Hopefully it will inspire some others to play with javac as well. It requires JDK 6. You can download the JAR file here and just run it with java -jar.
The source code is available in NetBeans CVS under contrib/SyntaxTreeNavigator (everything was reorganized for the recent migration to Mercurial, and I haven't figured out where it is in the new source structure yet).Update (3/31/08): I've set up a java.net project for the source code, syntaxtreebrowser.dev.java.net. It is pending approval but it should be publicly visible in a few days.
Caveats: (updated 2/29/07) - To use the application, you will need javac on your classpath. If you are running Apple's JDK 1.6 it already is. For other platforms you may need to run with
-Xbootclasspath/p:/PATH/TO/javac.jar (may be in tools.jar). - This application uses the
com.sun.source.tree API from javac. If you are not running Sun's javac, this package will probably not be there.
I have updated the JAR file to produce better diagnostic information, and added a source zip file.
Bookmark blog post:  del.icio.us  Digg  DZone  Furl  Reddit
Comments
Comments are listed in date ascending order (oldest first) | Post Comment
-
hi tim,
after I tried this on HelloWorld.java : just a static method, I did NOT see any output from the program.
Was there something that is expected of the user after pointing out the java source file through the dialog box ?
thank you,
BR,
~A
Posted by: anjanb2 on February 28, 2008 at 07:51 PM
-
Hello.
The application is looking for a missing class in my jdk6.
Exception in thread "AWT-EventQueue-0" java.lang.NoClassDefFoundError: com/sun/tools/javac/api/JavacTaskImpl
at syntaxtreenavigator.SyntaxTreeVisualizer.openFile(SyntaxTreeVisualizer.java:187)
at syntaxtreenavigator.SyntaxTreeVisualizer.openMenuItemActionPerformed(SyntaxTreeVisualizer.java:153)
at syntaxtreenavigator.SyntaxTreeVisualizer.run(SyntaxTreeVisualizer.java:62)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:183)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:121)
That is a very interesting utility, by the way. I have been wanting to play with AST for long. This might be a start once i make it work.
Posted by: pepe on February 28, 2008 at 08:58 PM
-
Hi Tim,
When downloading Netbeans 6.0 using CVS with the instructions here, If I use the -P option, I get no files at all and only a top-level file called MOVED that says everything's moved to hg. Without -P, I just get empty directories.
Isn't mercurial still "non-functional"? If so, seems like there's no good way to get the source and maybe you could just post a link to a file with the source for us :)
When trying to run it, I get a NoClassDefFoundError for com/sun/tools/javac/api/JavacTaskImpl class. Strange, because when I run it with -verbose:classes, I see that other classes (including JavacTaskImpl$2) are being loaded just fine from my tools.jar. I'm using JDK 1.6.0_10-ea with cygwin. I feel like a newbie again tracking down CLASSPATH problems.
Anyhow, I thought I'd mention it, as the other poster may be having the same problem.
Here are a few Java graph layout engines:
http://www.yworks.com/en/products_yfiles_about.htm
http://jung.sourceforge.net/pmwiki/index.php/Main/ImageGallery
http://www.jgraph.com/layout.html
Andy
Posted by: atripp on February 28, 2008 at 09:01 PM
-
Woot!
Just as I posted, pepe posted the same problem :) Nice to know it's not me and my CLASSPATH. Now I'm really curious as to what sort of stunt you're trying to pull on us at line 187 of SyntaxTreeVisualizer.java :)
Andy
Posted by: atripp on February 28, 2008 at 09:05 PM
-
Andy: Argh, I'll bet it's me and my mac. That is, Apple puts everything on the classpath; other platforms probably don't have javac on the bootclasspath. Adding it to your bootclasspath should fix it.
Also, if someone attempts to run it against the JRE, not full JDK, it's going to be unhappy.
Re hg, hg is working just fine.
hg pull http://hg.netbeans.org/main/
should get the sources. Out of date ones should still be available in CVS I think; you might have to pass a sticky date from before hg migration - the CVS server will stick around for some time because people still have work on branches and such, and history was not migrated for deleted things.
Sorry for any confusion :-/
-Tim
Posted by: timboudreau on February 28, 2008 at 09:17 PM
-
Oh, and thanks much for the links!
Posted by: timboudreau on February 28, 2008 at 09:18 PM
-
hi tim,
I tried this on HelloWorld.java :( just a static method), I did NOT see any output from the program.
Was there something that is expected of the user after pointing out the java source file through the dialog box ?
thank you,
BR,
~A
Posted by: anjanb2 on February 28, 2008 at 09:43 PM
-
Very interesting post, I will try what you wrote here.
MyBlog:
Posted by: sunil_gupta20801 on February 29, 2008 at 02:00 AM
-
it throws NPE when I try to open a file:
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at syntaxtreenavigator.SyntaxTreeVisualizer.openFile(SyntaxTreeVisualize
r.java:179)
at syntaxtreenavigator.SyntaxTreeVisualizer.openMenuItemActionPerformed(
SyntaxTreeVisualizer.java:153)
at syntaxtreenavigator.SyntaxTreeVisualizer.run(SyntaxTreeVisualizer.jav
a:62)
Posted by: mariso on February 29, 2008 at 06:44 AM
-
Mariso: This would be caused by
ToolProvider.getSystemJavaCompiler();
returning null. Are you running with a Sun JDK or something else? AFAIK, that should not return null...
Anjanb2: A window should open after you choose the source file.
Okay, this is more difficult than it looked...
Posted by: timboudreau on February 29, 2008 at 11:09 AM
-
This would be useful when hacking the compiler. Each pass of the compiler modifies the AST (especially Lower.java), and it can be difficult to understand what's going on. You could hand the top-level node to this tool, which would run in a separate thread, and have "breakpoints" in the compiler code that block the main compiler thread, allowing the UI to browse the current state of the tree as various transformations mangle it.
Posted by: lkesteloot on February 29, 2008 at 11:23 AM
-
Based on what I read here the javac tree api is supposed to work only with the sun compiler. Could the errors reported above be due to the usage of non sun java compilers?
From the interview:
In addition, Sun is providing access to the compiler syntax tree, a feature relied on very heavily by the upcoming version of NetBeans. While this is not directly in JSR 199, we're providing that capability in subclasses of the 199 API. At the moment, this only works on Sun's compiler, but providing standardized access to a tree API is something we might want to consider in the future.
Posted by: plakhina on February 29, 2008 at 12:00 PM
-
hi Tim,
Can you put the tools' source someplace so that I can debug it in my IDE ? It would be nice if you could create a netbeans IDE project for the same.
Atleast a source jar will help. I'm using JDK 1.6._03. from sun.
Thank you,
BR,
~A
Posted by: anjanb2 on February 29, 2008 at 01:13 PM
-
Sources are now available in the attached zip file, and I've updated the JAR file to provide better diagnostic messages for the issues mentioned.
Posted by: timboudreau on February 29, 2008 at 03:34 PM
-
Hi Tim,
Including tools.jar in my bootclasspath as you suggested works. Thanks!
The tool looks very nice, especially for just getting started using the compiler API.
For anyone else who's just starting to use this API, it might help to realize that the official public API, JSR-199, contains almost all interfaces, such as CompilationUnitTree. These are in the com.sun.source.tree package, and the javadoc is here
But the Sun-compiler-specific classes that implement those interfaces, such as JCCompilationUnit, are all defined as inner classes of the JCTree class in the com.sun.tools.javac.tree package. You won't find the Javadoc for those classes anywhere, as they're "Sun internal", and you'll have to download the JDK6 source and run javadoc yourself. They're in directory /langtools/src/share/classes/com/sun/tools/javac
Posted by: atripp on February 29, 2008 at 08:41 PM
-
Tim,
Seems you can never make something cool available without getting hit with a bunch of RFEs...
You're exposing all the internals of the JC* classes. It would be nice to have a checkbox to let me only see the public JSR-199 compliant data. Should be a matter of looping through all the methods of the *Tree interface rather than all the public get() methods in the class.
Also, looks like you display the class name at each node in the tree. For example, in a class with no "extends" or "implements" clauses, I get two empty "ExtendsClause" nodes under JCClassDecl. One is from the getExtendsClause() method and the other is from the getImplementsClause() method. It would be nice to both the name of the method called ("getImplementsClause()") as well as the actual class name of the objects returned ("ExtendsClause").
I'll see if I can get time to make these enhancements myself this weekend.
Again, thanks for the very nice tool.
Posted by: atripp on February 29, 2008 at 08:49 PM
-
Hi Again,
I did write a replacement class that does the things I mention above. It's called JavacASTTreeNode, and it's a direct replacement for your "V" class. It displays method names rather than class types, so for example, you'll see "getExtendsClause()" and "getImplementsClause()" nodes in the tree, rather than two "ExtendsClause" nodes. Also, it only does introspection on the JSR-199 "Tree" interfaces, so you'll see the standard "ClassTree" rather than the sun-internal "JCClassDecl".
Anyone let me know if this interests you.
Andy
Posted by: atripp on March 01, 2008 at 04:01 PM
-
By all means, if you feel like sending me a patch, I'd be happy to incorporate it.
Posted by: timboudreau on March 01, 2008 at 06:21 PM
-
Tim,
"Last spring I set about learning javac's Tree API (JSR 199)"
The Tree API is not JSR 199. They are closely related (along with JSR 269) but not the same.
Posted by: brucechapman on March 03, 2008 at 08:21 PM
-
I am a student at the technical university of munich in computer sience. Most times we do development with java in classes. So this tool seems to be very helpful for understanding. I just downloaded it and will test it. Thanks for sharing.
Posted by: landscho on March 18, 2008 at 03:16 PM
-
FYI, I've created a java.net project for this code - https://syntaxtreebrowser.dev.java.net/ - it should be up in a day or two, once it gets approved. Contributions welcome.
Posted by: timboudreau on March 28, 2008 at 05:39 PM
-
Source code is now in cvs there.
Posted by: timboudreau on March 28, 2008 at 05:54 PM
|
java.net RSS Feeds
