|
|
||
Tom Ball's BlogCommunity: NetBeans ArchivesElephants and the JavaFX Script CompilerPosted by tball on November 10, 2007 at 04:42 PM | Permalink | Comments (4)I don't know what it is about elephants, but they sure work well in analogies. Whether we are ignoring them, seeing pink ones or trying to sell white ones, elephant analogies seem to communicate ideas well. An elephant analogy came to me when I was recently grilled about exactly when the JavaFX Script compiler team will deliver our first milestone release. "I can't give you an accurate date," I said. "It's like pushing an elephant through a door*; until a critical mass makes it past the threshold you just don't know when you'll be finished. Once you pass that threshold, though, the rest happens quickly and in a manner that can be more accurately predicted." And then the tension in the room broke, as I think everyone has been in that situation sometime in their respective pasts. In a way, this compiler project has turned out to be very similar to some of the GUI toolkit ports I've worked on in the past. We're making lots of measurable progress, which anyone can monitor in our Another similarity this project has to a GUI port is that most graphical toolkits have a few core components that depend upon most of the rest of the toolkit; call those classes Widget, Component, JComponent, Window, (graphic) Node, it doesn't matter -- there are so many benefits to sharing code between GUI components that these elephant-sized classes are just part of the GUI toolkit territory. So porting a GUI toolkit means that often there will an excruciating period where there is little visible progress as you work through all the dependencies, and then your first window appears, and then most of the remaining toolkit seems to just fall into place quickly. That's what happened many years ago when I was part of a team porting Presentation Manager from OS/2 to CTOS (which was about as unlike OS/2 as could be possible in those days). For several weeks, we slogged along without any visible progress (as voiced daily by our frustrated management), then we finally got the login panel showing and, in less than a week, all of the core applications were up-and-running. That login application was the critical part of our elephant, since the closure of its toolkit requirements encompassed most of the system. It wasn't really the login panel itself, of course, because whatever application is chosen first will be the hardest. This is all a long-winded way of saying that we are really close toward delivering our first milestone release, but when the exact date will be remains unknown. If anyone is interested in monitoring our progress, check out our JIRA issue tracking system (courtesy of Atlassian), and/or subscribe to our developers alias (comments from experienced elephant pushers are welcome). Once we hit this first milestone, we'll be able to provide a much better road map for the remainder of this release. *No elephants were hurt during the writing of this blog entry.
The Birth of the JavaFX Script CompilerPosted by tball on July 20, 2007 at 06:49 AM | Permalink | Comments (3)Today a new JavaFX Script compiler incubator project, openjfx-compiler went live. Yes, it has source code, issues, mailing lists and a wiki, but the compiler itself actually does very little right now (although Chris Oliver did offer a nice tease). So why open it now? The answer is that our team wants to open not just its source, but the design and development of the compiler itself. We are also asking for feedback on the JavaFX Script language to make it fully specified and compilable. If you have an interest in watching or participating in a compiler for JavaFX Script, please join our project and help shape its future. If the past couple of months is any indication, it's going to be fast, furious, and fun! "Advanced Java Refactoring" JavaOne Slides AvailablePosted by tball on May 10, 2007 at 10:10 PM | Permalink | Comments (1)Today I gave a talk at JavaOne titled "Advanced Java Refactoring: Pushing the Envelope" to a packed room, and just pushed a PDF of the slides to the Jackpot project site. Unfortunately I mismanaged the talk's time, spending too long on the demo and had to blast through the last (and, IMHO, the most interesting) slides. My apologies to those in the audience for not doing a better job by them. It was great to see the continued interest in Java refactoring as its state-of-the-art is improving so rapidly among all the Java IDEs. I've come to rely on the 6.0 NetBeans Java editor over the past several months, in large part because many common refactorings and some new ones are fully integrated into the editor, instead of hiding in a separate Refactoring menu. Traditional refactoring improved my productivity when I first started using it many years ago, but this tighter integration is like strapping booster rockets to my chair. So my talk wasn't about the cool refactorings I'm starting to work on, but instead about how Java tools are changing the very meaning of refactoring and taking it much farther than it was first envisioned. When debating language strengths, one thing Java engineers may be overlooking is how much Java's being a strongly-typed and statically inspectable language determines how refactorable it is. For example, the bytecode verifier that has been a part of the JVM since 1.0 puts several limits on what is considered valid code, since the verifier has to model that code accurately in order to verify it. Therefore to a certain extent, any language that can run on the JVM can be modeled fairly well. But Java adds a lot of semantic information which makes much richer models possible than with most other languages, so that language-aware tools like refactoring editors have much more information to work with when working with Java. It also avoids features that undermine modeling, such as C preprocessor abuse. Lately I have been disappointing Sun's C and JavaScript tools engineers who want to leverage the NetBeans IDE's new Java refactoring engine (much of which is based on Jackpot), because the only answer is "no can do" -- their languages can't be statically modeled as completely as Java can. So if you get into a late-night pub discussion about how Java is so last century and that dynamic languages will soon rule the world, pick some of your favorite refactorings (those that really improve your productivity) and ask if their language/IDE supports them. Now, there are some brilliant people working on world-class languages, but they will have trouble competing with engineers leveraging the rich Java models generated by javac, Jikes, IntelliJ IDEA, and other Java tools. I suspect improved refactoring is one way Java will remain viable for many years to come. My JavaOne Talk on Advanced RefactoringPosted by tball on March 02, 2007 at 11:19 AM | Permalink | Comments (3)Chris Adamson was complaining that no one was announcing that their JavaOne talks were accepted, so here goes: my talk, "TS-9861: Advanced Java Programming Language Refactoring: Pushing the Envelope", was just accepted. It will both be about some of the radical transformers I'm currently working on, but more importantly about how the current state of tools technology can be used by other developers to push the refactoring envelope further. The small matter of backing my assertions with some working code has been the reason for my recent silence. Why aren't more people announcing whether their talks were accepted or not? I can think of several possible reasons:
Jackpot just open-sourced? But I thought it already had...Posted by tball on January 25, 2007 at 03:31 PM | Permalink | Comments (7)After months of slogging through Sun's Open Source Review process, last night I committed Jackpot's source to the NetBeans public CVS repository. If you want to browse it, follow the directions in the NetBeans CVS Access page, specifying "jackpot" as the module to check out. You may be in for some disappointment if you do this, however, as all these source files provide is IDE integration for Jackpot, its UI, and rule file editor support. Where's the beef? Where is the code that does pattern matching on compiler ASTs and transforms them? Where is the source rewriter? WHY IS SUN YANKING MY CHAIN AGAIN!!! (I really get email like this occasionally.) Now take a deep breath and relax. The truth is that Jackpot started life as a research project prototype which found some useful conclusions, including:
The Jackpot piece most interesting to compiler junkies was released a few months ago. The new Java editor, which is coming in NetBeans 6.0, is based on the Compiler API and the transformation engine from Jackpot. That engine was quietly included as part of the new editor's release to open-source last October:
To browse the engine's source (and the Java editor enhancements), check out the "java/source" project using the CVS instructions above. Yesterday the remaining parts of Jackpot were released to open-source, the ones no other project needed to publish first. The project is now a lot smaller than when we first transferred to the Developer Products Group (aka Tools), but its integration into the javac and NetBeans Java editor source bases will ensure its wider use throughout the Java community. Yet Another "5 Things" BlogPosted by tball on January 16, 2007 at 11:55 AM | Permalink | Comments (2)Here I was hoping to avoid participating in the "5 Things" blog game, but then Greg tagged me and sent an email vaguely warning against breaking the chain. So here goes:
Get Medieval On Your CodePosted by tball on November 20, 2006 at 02:22 PM | Permalink | Comments (6)I realize I'm dating myself paraphrasing 1994's "Pulp Fiction", but it really pays to get and stay hard-core with your project code. One easy way to turn up the heat is to enable all compiler warnings and tell the compiler to treat them as errors -- all of a sudden, those "minor" cleanup issues one never has time for now demand immediate attention. This actually save time in the long run, because once you go through the work of cleaning up all the existing warnings, any new problems can be quickly caught while the new code is being developed (rather than having to scramble after the product ships). Serious C developers have been doing this for years; the Solaris team, for example, bit the bullet maybe fifteen years ago and went "lint free", running the lint tool as part of its build and addressing all warnings it and the compiler reported, then requiring that all new code pass this same hurdle. Even though it was a major effort, that cleanup has paid major quality dividends ever since. In the past few years, Java has finally started to catch up with C on this front. Third-party static checkers like FindBugs and PMD are now freely available and are easily integrated into a build. In Java 5 the javac compiler added the -Xlint option with several types of checks, which were significantly expanded in Java 6. The Jikes compiler has a similar option called pedantic compilation (+P). All of these tools allow you to enable/disable different types of tests, and unless you strongly disagree with certain tests I recommend that you turn on all the warnings you can. Sometimes you need to suppress one or more warnings for a specific block of code. Java 5 introduced the SuppressWarnings annotation to enable you to do this, but javac ignored it in that release; happily, it is now recognized in Java 5, update 6 (1.5_06). One big advantage of globally checking for a type of potential error and suppressing it in specific places is that the SuppressWarnings annotation serves as documentation that you really intended to do this questionable practice. For example, Jackpot's parser outputs a big switch statement for a rule set, which is optimized using case statement fall-through. Since fall-though can be a frequent source of bugs, Jackpot is tested with javac's fall-through checking and with a @SuppressWarnings("fallthrough") for this specific method -- that way if there are fall-through bugs elsewhere they will be flagged. Turning warnings into build-stopping errors has also been enabled in Java 6 via javac's hidden -Werror flag (just like the C compiler's). Jikes has the +Z2 flag to do this, and recently it added a -Werror equivalent to be command-compatible with javac. For NetBeans projects, it is easy to set these flags by right-clicking on the project, selecting its properties, clicking on the Compiling panel and putting them in the "Additional Compiler Options" field. It can be a pain changing all of your Ant build files to do this, however. You can set this property globally, however, because NetBeans projects have a javac.compilerargs property hiding in the nbproject/build-impl.xml file we can use. To set this for all projects which have this property, add this line to $HOME/.antrc (which probably doesn't exist): ANT_ARGS='-Djavac.compilerargs="-Xlint:all -Werror"'The Ant startup script reads this file and passes the ANT_ARGS property to Ant's main class. If your project wasn't created by the NetBeans IDE, you can still edit the build.xml file and add this property to the javac task references. Separate static checker tools can be integrated into your build as well. Ant tasks are available for both FindBugs and PMD. Both tools have a failOnError parameter which can be set true so the build stops if a warning is issued. Both of these tools support the SuppressWarnings annotation, too. Now, those of you who are really hard-core will point out there are many other ways you can stop coddling your build and make it cough up its bugs. You are right, and in future blog postings I'll investigate other ways to get medieval on your code. Hacking Open Source javac!Posted by tball on November 13, 2006 at 01:20 PM | Permalink | Comments (1)Hallelujah! Java SE is now an open-source project, with the HotSpot VM and javac as its first sub-projects. This is great news for anyone interested in Java performance, as the world now has full-access to both the static (javac) and dynamic (HotSpot) optimizing compilers from Sun's Java implementation. Remember back when everyone laughed at the prediction that with dynamic compilers like HotSpot, Java performance will eventually surpass C++? This is starting to happen now with the continued performance gains in Java 5 and soon Java 6. My hope is that an open-source HotSpot and javac will encourage the world's best and brightest to augment the excellent work of the JDK team to drive Java performance further. So how can you start contributing? The first step is to download and build the source using the links above. But wait! There is an easier way to work with javac's source: NetBeans has a brand-new nb-openjdk project which makes downloading and building the javac source painless and productive. My guess is that this project will grow to support the HotSpot project as well as the rest of the JDK's components as they are released. Happy hacking. Hacking javacPosted by tball on September 16, 2006 at 03:55 PM | Permalink | Comments (20)hack (hăk) n., A non-obvious solution to an interesting problem. This definition is on the front of a tee-shirt I have from O'Reilly Media to promote their Hack Series, which includes one of my favorite books, Swing Hacks by fellow java.net bloggers Joshua Marinacci and Chris Adamson. The reason I like the Hack Series is that even for subjects you know fairly well, these books describe interesting solutions which I hadn't realized. It's that constant discovery of "non-obvious solutions" which has kept me so interested in programming over the years. Take compilers, for example: they just compile source code into object code, right? Hacking them is generally frowned upon because as Ken Thompson discused in his ACM award acceptance speech, Reflections on Trusting Trust, they are a great place to hide a trojan horse. But if the compiler is designed to function as a tool library, as javac is, much more interesting (and benign) hacks are possible. Mustang (excuse me, Java 6) has three related API which all Java tool hackers should check out: JSR-199: JavaTM Compiler API, JSR 269: Pluggable Annotation Processing API, and the Tree API (com.sun.source.tree and com.sun.source.util). The Compiler API is deceptively simple: it gives you the ability to programmatically invoke javac (actually, any tool that implements the Tool interface such that ServiceLoader can find it) and compile one or more source files to class files. That's nice if you are writing an appserver container, perhaps, but not very hack-inspiring, right? As Robin Williams said in the Disney cartoon Aladdin (I have small kids), "Wrong! But thanks for playing." The first hack leverages JSR-199's DiagnosticListener interface, which lets you listen to the errors and warnings created by the compiler. NetBeans uses this technique to display errors while you are editing Java sources; compilation is run regularly in the background but only the diagnostic events are used. JSR-199 improves on the old trick of parsing error strings with Diagnostic instances, which provide accurate source position information, and locale-independent error IDs with locale-specific error text. Still bored? How about creating your own scripting language? Rather than write a complicated intepreter, write a (hopefully) simpler compiler which outputs Java source files. Then use the Compiler API with a custom ClassLoader to dynamically load these classes on-the-fly, as if they were interpreted. Think the process is slow? The Jackpot rule language parser generates Java sources and uses this hack to compile them (look in $HOME/.jackpot). We used to conditionally compile scripts only if they had changed, but found that javac is so fast that caching didn't make a difference (we just keep the files for troubleshooting). If you don't want to write anything to disk, a related hack involves implementing the JavaFileManager interface to use memory instead of files -- javac doesn't care about files since it only uses streams supplied by whatever JavaFileManager implementation you provide. JSR-269 is officially the "Pluggable Annotation Processing API", and while it does that very well it also enables lots of other hacks. A general way to think of JSR-269 is that it gives you access to all of the types and elements (symbols) in any set of source files, not just its annotations. Its javax.lang.model.util package has easily extendable visitor and scanner base classes which can be used by tools to inspect projects at the semantic level. One group of tools which can use this hack include error checkers that validate design rules, such as that any class which extends Object.equals() also extends Object.hashCode(). One advantage of using JSR-269 types and elements rather than your own parser for these sorts of tests is that while you can infer many properties from a parse tree, the javac semantic model knows the correct properties. If you want to dig deeper than types and elements, the Tree API makes this sort of hacking simpler. Because the Tree API is not defined by a JSR, the JSR-199 and JSR-269 APIs cannot directly refer to it. So if you just look at those API, it seems impossible to inspect class members beyond their declarations. The semi-secret hack here is that javac's javax.tools.JavaCompiler implementation returns a CompilationTask instance which is also an instance of com.sun.source.util.JavacTask. This class is much more interesting to tool hackers, since it give you access to the parse trees and all JSR-269 information, plus control over javac's execution. Don't need to generate class files? Just invoke JavacTask.analyze() instead of JavacTask.generate(). This class also provides access to the Types and Elements utility classes, which make hacking even easier. So if you know you are using javac as the tool provider, you can just cast the task instance to JavacTask and have fun. These API provide a read-only model of Java source code, so it is difficult to modify source code programmatically (you can overright the files, of course, but comments and formattting get blown away). For hacking source files without angry mobs coming after you, you'll want the Jackpot API, which extends these API to provide model transforming and formatted source rewriting. These four API work together to define a toolkit to create just about any Java language-aware tool. What sort of "non-obvious solutions" can you create with them?Is Writing Code a Career Limiting Move?Posted by tball on August 19, 2006 at 11:02 PM | Permalink | Comments (20)A CLM, or Career Limiting Move, is an action on your part that threatens your promotion chances, or even your job. Regardless of whether you have heard of a career limiting move, most of us know one when we see one; serious ones include harassment or ethical violations, as well as less severe ones like backing your car into your manager's new Porsche or mocking the VP's favorite sports team without realizing who's eating lunch at the next table. Engineers may joke about something being a CLM, but real ones are to be taken seriously. I love to write code and am a big believer in the Agile software development model. Most Agile software methodologies require that the design grow from an initial prototype which gets refined based on early and frequent user feedback. In theory, the project lead can specify a design which someone else prototypes, but the user feedback is much more likely to be heeded and integrated early (when it can have the biggest impact) if the project lead is actively involved in the prototype's code. To date, my CLMs have been minor, such as avoiding useless meetings. But recently it appeared that my love of programming was a more serious CLM, as it seemed to undermine my credentials as a senior software engineer. How? The top jobs these days are for "software architects", and architects aren't supposed to write code; instead they write specifications which us lowly programmers are to blindly follow. This was certainly true with the J2EE engineering team a couple of years ago, where the "spec lead" was not allowed to write any code; that was the job of the junior "implementation engineer" for that API. I got into a heated exchanged with one of their Distinguished Engineers over this issue, and of course arguing with DEs at Sun is yet another CLM! I doubt that practice is true these days, however, as Java EE 5 is so much easier to use it's obvious that coders must have actively participated in its design. I've been reading a great book from the Pragmatic Programmer group called Practices of an Agile Programmer. One of its chapters is titled "Architects Must Write Code", which includes this gem: "Real insight comes from active coding." Tim Lindholm, Distinguished Engineer and Java ME Architect, put it more sucinctly eleven years ago when we worked on the JDK team together: "there are coders and there are bullshitters*." (I hope he is still coding now that he is an architect!) I think this "architects write specifications, not code" is a holdover from the discredited Waterfall model of software development, where systems analysts fully specified a project before coding began. While this model may have made sense in the sixties writing COBOL (though I doubt it), I don't understand how it is able to remain entrenched in development organizations forty years later. The good news is that many companies will acknowledge and correct their biases when you make them known. I recently complained to my management at Sun about this subject, expecting at best silence on their part. It was quite a shock to see them turn on a dime, reaffirm the importance of coding for senior developers, and make changes that addressed my personal career concerns. So if you are in a similar situation where it looks like your coding is no longer appreciated, discuss the problem with your manager. You may be surprised to find coding's "career limiting" is all in your mind! *I think it's okay to say bullshit on java.net, as it's an acceptable word for intellectual discourse since the Princton University Press published On Bullshit. It's a great book; read it to learn more about why architects who don't write code can be so successful. :-) ¡Hola, del Mar Caribe!Posted by tball on June 26, 2006 at 09:47 AM | Permalink | Comments (1)This is week two of my forced vacation at Cancun, Mexico; this week we are staying at the Grand Mayan near Playa del Carmen. Forced? This may come as a shock to some, but vacation is hard to take for many engineers including myself. On the recommendation of another work-aholic engineer, my wife and I bought a Mexican vacation timeshare where I cannot work or access the Internet reliably. Because she is frugal (not that I'm complaining!), she will not allow us to pay the yearly maintenance without getting a resort vacation for our money. So I go on vacation each summer kicking and screaming (internally), but find it does me a world of good to decompress. And the scenery is nice, too. Ah, but getting away from it all is getting harder every year as the Internet expands to encompass our world. We brought cellphones to keep in contact with each other (it's a big resort), but that means that people can reach us from California just as easily. Cyber-cafes were something I used to have to track down in the wilder parts of town, but now the resort itself lists a cyber-cafe as one of its amenities. The hotel during our first week in Cancun had free wireless access, and my guess is in the next year or two all resorts will. A few nights ago I found a guy sitting in the hallway outside of the model suite here, because in roaming around with a WiFi sniffer, he found an open hotspot. Soon, he won't have to hunt for one. In spite of the expanded network coverage, it's still great to move a little away from city life. There is Yucatan jungle on both sides of our resort, with a sign that clearly states "No molestar los cocodrilos, por favor." (Google can translate this for you.) As Dorothy said in The Wizard of Oz, "I don't think we're in Kansas anymore." So wish me luck as I try to make the rest of the week. When I get back it will be full-time coding again, but I suspect that I'll still hear the ocean's waves breaking in the background for quite awhile. See You at JavaOnePosted by tball on May 12, 2006 at 11:53 AM | Permalink | Comments (3)It's been a crazy couple of months, but JavaOne waits for no engineer. Most of Jackpot's new UI has been implemented and was pushed yesterday. This spec required fundamental changes in the transformation engine, which I think strengthened its design even though it impacted the schedule. It's also more Swing form coding that I have done in a long time, but thanks to Matisse I just made the JavaOne milestone. If you are interested in Jackpot, there will be an announcement regarding it at NetBeans Day. Thursday morning I'll be presenting Creating and Deploying Custom Jackpot Queries and Transformers (TS-1278), which will describe how developers can create their own bug-searching and fixing tools and share them with others or deploy them within an engineering organization. Friday, there will be a short Jackpot demo at James Gosling's keynote. (Looking up the URL, I just found out Scott McNealy will be hosting that keynote, too. I better "kick butt", as he likes to say, during the demo!) If you are attending JavaOne, please check out these "Jackpot sightings" if you can. If you see me rushing down the hall, join the rush and introduce yourself. A Little LanguagePosted by tball on March 22, 2006 at 01:35 PM | Permalink | Comments (3)I have been getting some comments regarding the limits of Jackpot's rule language which suggest that Jackpot itself is not very powerful or useful. I must therefore be at fault for not making it clear that the rule language is a "little language"; it was never meant to be a complete transformation language for Java, but instead was designed to be an easier way to describe many of the more common types of transformations possible with Jackpot. Mark Grand, the author of Patterns in Java, Volume 1, A Catalog of Reusable Design Patterns Illustrated with UML, describes the Little Language design pattern this way: Suppose you need to solve many similar problems and you notice that the solutions to these problems can be expressed as different combinations of a small number of elements or operations. The simplest way to express solutions to these problems may be to define a little language. Common types of problems you can solve with little languages are searches of common data structures, creation of complex data structures and formatting of data.
While the Jackpot API can be used for all kinds of source transformations, the rule language was created to facilitate conversion of large sets of methods, such as updating projects that use deprecated methods in the Java core libraries. The general rule is that a programmer will do the same task several times, but will soon write a program to do it before he or she dies of boredom. The AWT alone has over a hundred deprecated methods, so multiple the writing of 10-20 lines of code for each method and you'll see why the task quickly lost its appeal! Using a table to map old to new methods was considered, but that idea was quickly discarded because frequently the method signatures were different (a simple example is converting " Component.show() to Component.setVisible(true)").
So a little language was created which uses Java fragments along with meta-variables (wildcards) to transform statements, along with guard expressions to utilize the type information in the underlying model to restrict transformations safely. It may seem initially confusing since the language looks a lot like Java, but once all those dollar signs are explained, most Java developers immediately grasp its syntax. I hope any enhancements to this language keep the current simplicity. Many people have asked why a pattern language such as XPath wasn't used. There are two reasons; the first is a minor one in that the rule language doesn't require new developers to learn a large set of tree node types (the Tree API has forty-nine). The second reason is more important: Java isn't a single language, but a progressive set of languages with subtle (such as assert statements) and not-so-subtle (generics, enums, annotations, etc.) enhancements between versions. The "-source" flag your project passes to javac actually tells it what sort of syntax tree its parser should create, as those trees can be different for different source values. So an XPath expression will work fine for single tree fragments, but the more elements in your path, the greater the chance that your expression won't match as expected. Jackpot uses the project's source level when compiling the rule file, so potentially different compiled matchers will be applied to projects with different source levels. So if the rule language has these limits, what good is it? If you have installed and run Jackpot, the answer is hiding in your home directory where a ".jackpot" directory has a cache of the Java and class files created when a rule file is compiled. If you browse the generated Java files, you will see that all the rules are combined into a single switch statement; this allows matching to be done in a single pass so rule files scale well for large rule sets and large projects. It saves a lot of coding, too: SimplifyBooleanExpressions.rules, for example, has 45 rules which generate to a 657 line Java file. Based on feedback the language has grown to what it is today, and we encourage more feedback on how you think it can best be enhanced. And if you have a great idea for a transformation which the rule language doesn't support, don't give up! The Jackpot API lets you create it without rule file limits, as these examples demonstrate. Plus, it's a whole lot easier to code-review a rule than its equivalent compiled code. The Power of CommunityPosted by tball on March 17, 2006 at 02:54 PM | Permalink | Comments (0)I've been a big fan of early community involvement in projects, especially after how much it helped drive Swing to its first release. Back then the general rule was to post an alpha, silently wait for feedback to trickle in, ship a beta months later, wait again, and then release the batch without any response to most of the feedback. Swing broke that process by releasing snapshots every two weeks (maybe that's where the Mustang folks got the idea!) and answered the hundreds of emails that helped improve those snapshots. It was exhausting, but I seriously doubt Swing would be what it is without the strong involvement of those engineers. This week I was happily reminded of just how powerful community involvement can be. An early access release of Jackpot went out and Tor Norbye blogged about it within minutes of the announcement on the NetBeans development email alias. Tor included a screen snapshot of some sample Jackpot transformation rules, and within a day several engineers commented on a problem with one of the rules. I then got email from one of the commentators (Eylon Stroh) pointing that the issue could be fixed with two rules he listed; he then noted: Although, what would really be neat is
{ $p$; $v = $e; return $v; } => { $p$; return $e; } :: declaredIn($v, $p$);
which seems possible in principle, but not with the current rules language.
The reason for his comment is that the rule language didn't have a "declaredIn()" expression. So that expression has been added, as described here.
There were two things I found really cool with this experience:
Jackpot Early Access AvailablePosted by tball on March 13, 2006 at 05:03 PM | Permalink | Comments (11)![]() Well, it's been a long haul, but Jackpot is now available as an early access release. It's not pretty or have a rich command set yet, but the core engine works and there is lots of documentation to read and sample transformations to try out. Everything can be found at the Jackpot project home page. As previously promised, this early access release is geared toward developers who want to create their own queries and transformations, rather than people who just want to use built-in commands. For the JavaOne timeframe, we are planning on a beta release with some major UI improvements as well as a much more interesting command set. I'm looking forward to reading everyone's feedback. If you have ideas about Java transformations, join the project! Jackpot's Coming (Finally!)Posted by tball on March 03, 2006 at 11:05 AM | Permalink | Comments (7)Patience is a virtue? I never really learned that growing up, and impatience has served me and most of my fellow developers much better in our careers. So it's been a frustrating time since JavaOne lining up my ducks (an expression which probably doesn't translate well, but visualize how hard that would be) to make Jackpot available. Yesterday we had a big breakthrough, though, with marketing buy-in for a phased release plan. "O frabjous day! Callooh! Callay!� He chortled in his joy." There were two important requirements for a Jackpot release: provide a public, stable API that doesn't reference any javac internals, and fully integrate it into the IDE environment. The good news is that the read-only Java model developed by the javac and apt teams is now almost finished and Jackpot's API has been converted to reference it. Yes, the JCP process is painfully slow for those of us who lack patience, but both teams have delivered great work that should stand the test of time. I have been moving more slowly on the IDE-integration front, but things are looking up. Because of the focus on rewriting the engine to use the new public APIs, I haven't had time yet to write any killer transformations. The early release will therefore just have sample commands which demonstrate Jackpot's potential, so expect some disappointment when you first try it. Its UI is also much too modal: you start a "session", invoke queries and transformers, then end the session to save the changes. To make matters worse, if you edit any files in the middle of a session, the session is invalid but you won't be notified of that until you try to save. My apologies for creating such a bad interface, but it's temporary. This week's other good news is that because NetBeans 5.5 is now in preview, I will finally get some help from the overworked NetBeans UI team. We've only had one meeting so far, but I left with several excellent ideas for better integration of Jackpot and thus a better UI. Let my mistake be a warning to others: just because you are a good coder (or even a former Swing architect) doesn't mean you are a good UI designer. I never thought I was very good at design, but this meeting reinforced that awareness of my limits -- I guess once a systems engineer, always one. So the current plan (don't call it a schedule) is as follows:
NetBeans 5.0 is Making Me GrumpyPosted by tball on September 28, 2005 at 01:28 PM | Permalink | Comments (2)It's an old comic routine, where a grumpy old man complains about how much easier "kids today" have it than he did. Personally I think Dana Carvey perfected it in his Saturday Night Live character, but I bet most of us have at least one older relative who sounds like this guy at times. With the way software development changes so rapidly, it's easy for any of us with a few years experience to sound grumpy when some newbie complains about how hard programming is. "You think NetBeans is hard? Why in my day we used Emacs, and even though it made our fingers ache we loved it!" ("You had Emacs? Back in *my* day we used punch cards, and we were glad we had them." "You had punch cards?" and so on...) The NetBeans 5.0 beta just released, with a slew of new features that improves application development and therefore, can make old-timers grumpier. The feature that makes me grumpy is one many NetBeans users won't care about: developing NetBeans Modules and Rich Client Applications is now much, much easier than in previous releases. Core tasks that once took me days to learn are now boiled down to clear, easy-to-understand wizards. Extending NetBeans is no longer limited to the core set of engineers dedicated (and masochistic) enough to devote the necessary effort to use this API effectively. As a recovering engineering manager, it's been a pleasure watching the "apisupport" team design and implement this critical feature. First, a small team was created and, as much as possible, dropped out of sight from the rest of the engineering group. As NetBeans management learned several releases back, rewriting core functionality requires isolating the team from corporate antibodies, which can kill innovation that threatens existing systems. Next, NetBeans lucked out and hired an exceptional doc-writer, Geertjan Wielenga. I normally don't single out individual team members for praise since engineering is a team sport, but Geertjan's contribution was critical because he didn't function like a normal doc-writer. Maybe it only is true at Sun, but here most writers are forced by their heavy workloads to avoid getting too involved in early product development, preferring to wait until the design is finalized so as to avoid having to drastically rewrite their work. Geertjan, on the other hand, jumped right in with no previous NetBeans API exposure (as far as I know) and quickly started writing several short tutorials (found here), bombarding the developers with questions and then forwarding frequent drafts to the team for review. Because the development engineers were trying to make NetBeans API development easier, his questions helped shape the various wizard designs (if a developer has to ask too many questions, the current design isn't working). Other equally good doc-writers might ask similar questions yet don't impact the product's code as much, because they waited until the design was frozen when they started writing documentation for it. Geertjan's tutorial code also showed lots of places where NetBeans was unnecessarily burdening module developers by requiring hand-written, boilerplate code. The development engineers therefore reviewed each tutorial with an eye for how they can shorten or even eliminate the tutorial with a wizard design, because documentation also functions as design criticism (i.e., good designs shouldn't require much explanation). Geertjan then quickly made massive rewrites and eliminated a lot of code as the API support improved. His impact on the project therefore can't be weighed by how many words made it to the public website. I wish more doc-writers and QA engineers were actively engaged in the initial design and development process, but suspect it is a management scheduling problem rather than a lack of interest or commitment.
My biggest complaint? I wish this work had been done six months ago, when I started working on making a NetBeans module which uses Jackpot, as it would have saved me literally weeks of work. So be warned that I may get grumpy about complaints about how hard module development is. <rant>"It took you a week to extend NetBeans 5.0? Why that's nothing! In my day we spent months staring at source files 'til our eyes bled, but we were happy to do it."</rant>
Java 5's Varags Clean Up Reflection CodePosted by tball on September 24, 2005 at 11:09 AM | Permalink | Comments (4)Java 5 has a lot of major improvements to the platform, to the point where some of the smaller features can get overlooked. Like many developers (including Calvin Austin his J2SE in a Nutshell article), I thought it great that varargs methods were added so printf() could be implemented, but didn't see much other use for the feature.
Yesterday I was updating a Factory class which uses a ClassLoader and Java's reflection facility to separate a public interface (JackpotEngine) from its implementation (Engine), and got an unexpected compiler warning about vararg misuse calling Can't Tell the Forest for the TreesPosted by tball on September 20, 2005 at 08:39 AM | Permalink | Comments (3)One big reason Jackpot hasn't released yet is because it is tightly bound to javac's private modeling API for abstract syntax trees (parse trees), types and symbols. The two teams have been working together on how such an API can be exposed without locking down javac and preventing future innovation. The solution the javac team designed is an interface-based AST abstraction, which javac will implement while still keeping its internal implementation private. This, along with the proposed enhanced Mirror API that is part of JSR-269: Pluggable Annotation Processing API as well as JSR-199: Java Compiler API, will be the only javac API that Jackpot developers will use. Interested? Peter von der Ahé has a blog entry describing how to join Tree API project. See you there! JFind: a simple jar file locatorPosted by tball on September 03, 2005 at 01:24 PM | Permalink | Comments (13)I'm a big fan of the Unix find utility for managing large projects. One task that's difficult to do with it, however, is finding which jar file contains a Java class -- you can't use search commands like grep because it will match on any class that references your target, not just the defining class itself. There are several utilities that do this on the web, but most are big GUI-based tools which seem like overkill for what should be a small job.
So a while back I wrote a simple command-line utility called JFind, which is now in the NetBeans repository here. It's a single, simple class, which relies on the java.io and java.util.zip packages to do the heavy lifting. Just specify the class or package you want to find and a directory to start your search, and it will report the jar file or path that defines it.
For example, to find which jar provides org.openide.ErrorManager in a
NetBeans distribution, run:
java -jar jfind.jar org.openide.ErrorManager dist
To find which jar provides the javax.crypto package in the JDK:
java -jar jfind.jar -p javax.crypto JDK-directory
One could write a more complex, Swiss-army-knife-like utility (like the Unix find utility), but sometimes short and sweet works best.
Project Jackpot at JavaOnePosted by tball on June 15, 2005 at 11:35 AM | Permalink | Comments (11)My apologies for not writing recently, but it has been for a good reason: I will be giving a JavaOne presentation on Project Jackpot: A New Java™ Technology for Source Transformation, and have been working hard on a NetBeans plug-in to demonstrate it. I am really excited to be able to finally discuss our work to such a wide audience. What a long, strange trip it's been! James Gosling and Michael Van De Vanter started the Jackpot Project at Sun Labs back in the spring of 2000, and I joined a month later. For the first three years, "Jackpot" was just the project name while we investigated different ways tool improvements can enhance Java developer productivity. Rather than publish papers with our ideas, we made the radical decision (for Sun Labs, anyway) to publish our results as open-source NetBeans modules. The metrics (now obsolete) and classfile modules are a couple of Jackpot "publications". In late 2002 we focused on an experiment James developed which read Java source into a model, applied transformations to that model, and then wrote the model back to source files. What was different about his approach is that its model allows a tool to make much more radical (but safe) changes to Java source code than is possible when working with syntax trees. The need for a deep understanding of the Java type system in a source transformation tool became even more apparent when generic types were added in Java 5. Project Jackpot therefore uses every scrap of information it can pull from javac rather than try and create its own parser, as it is a huge effort to correctly model all of the subtleties of the Java language.
When James became Sun's Chief Technical Officer of the Developer Products Group, the rest of Project Jackpot transferred over as well to the NetBeans core engineering team. Jackpot was put on hold so that javac's parser and error-checker could be leveraged by the NetBeans refactoring engine and editor. Once 4.0 released, the Jackpot engine has been re-designed and re-written (hey, it was just a research prototype!). Now that this rewrite is fairly stable, my top priority is incorporating Jackpot into NetBeans so Java developers can start using it. And yes, the plan is to make it an open-source project, just like the rest of NetBeans. As my father used to tell me, there is no rest for the wicked. :-)
I hope you are coming to JavaOne and can attend my session on Tuesday, June 28th at 1:30. I will also be attending NetBeans Software Day on Sunday the 26th. After JavaOne I will be writing more often about Project Jackpot and the tips and tricks I learned while developing it.
| ||
|
|