Eitan Suez's Blog

Skip the Compile

It seems to me that many issues that come up in Java stem from the "I don't want to have to recompile my code" argument. All kinds of design decisions stem from it too. We end up with systems that are written in two or more languages. What I mean is that we usually end up with a mix of Java code, properties files, xml files and other stuff. The properties files and xml files can be viewed as interpreted mini-languages that bypass the Java compilation hurdle.

Many debates surrounding Java 5 annotations also are related to this issue: many have argued that they don't want to have to recompile their code to revise the value for a piece of metadata. Fair enough.

How many times have you heard or read of a selling point for a software framework or product or solution being "and you can make changes to the system without having to recompile the code." I must admit I'm sure I used that line more than once before (in my early days programming :-)).

So here's what I find interesting: instead of coming up with all kinds of schemes to get around the problem, why don't we just deal with the root cause? I think it's staring at us in the face: why don't we simply program in an interpreted environment?

I think we all must admit that it's much nicer to be able to make a change to a web page or a template or a piece of code and very quickly turn around and test the change without having to recompile. Compare that to the way a jsp works: compile the jsp into a servlet and compile the servlet into bytecode. It's true that servlet containers are easily configured to 'autoreload' jsps. So in this case, it's really no big deal.

But that's not what I'm talking about here. What I'm talking about is if we need the equivalent of a config file, we don't have to go out-of-band and step out of the Java environment and start our new metalanguage in a properties or XML file like James Duncan Davidson did with Ant or like we do every day. We would simply write source code. An entire thick layer of nonsense design decisions would simply disappear. No more debates about put this code here or there depending on whether we want to incur the cost of a recompile.

This essentially is the argument for internal domain specific languages (DSLs). So Rake files are not written in XML like Ant is but instead they're interpreted by Ruby, the same interpreter that you use to write your Rails app (note: ruby and rake and rails are discussed here as means to provide an illustration. this entry is _not_ about 'ruby is better than java' but rather about the more general idea that 'interpreted might be better than compiled, from this point of view anyway').

More and more these days I'm coming to think that a single or unified underlying model for interpreting stuff would be a good, pleasing thing. I like this kind of simplification. We end up with a more uniform system, and spend less time reinventing the wheel.

I find it most interesting that for some reason, discussions around this basic issue never seem to come up. What comes up are people talking about or trying to solve the symptoms of the root cause, but the root cause seems to evade us.

Note: this entry is cross-posted on my personal weblog

Comments

• I've thought for a long time that the VM should be able to load classes with .java extension, invoking javac at runtime. I think it would make development easier.

  Posted by: keithkml on January 17, 2006 at 08:02 PM

• Folks have been writing about it for a couple of years. It's too bad this fundamentally excellent concept hasn't received more press. For a growing number of things I've freed myself from the code/compile/deploy/test cycle.

  The best solution I've found is BeanShell. Two things about it are hugely important to me:

  1. It supports the Java syntax.

  2. It's interpreted, and you have the option of compiling to byte code.

  Because it's the Java syntax you can also package/deploy your app the old fashioned way with javac/ant if you like.

  What missing is a NetBeans module that lets you run your BeanShell code directly from the editor against an embedded Tomcat/Glassfish. No enormous amounts of time wasted for compile/deploy - simply code.

  http://weblogs.java.net/blog/joconner/archive/2005/11/javaone_tokyo_0_9.html

  Posted by: markswanson on January 17, 2006 at 08:14 PM

• But is that really what one wants to have with a typed language? Generics have been introduced to enable the compiler for even stronger typechecking than it used to be. I admit that compile time errors caused by modules one depends on are annoying, but runtime errors are worse. I don't want to miss the errors and warnings a compiler gives to me. Another point is to inject the changed code into a running application; this facilitates debugging/fixing applications having huge startup cost or deployment needs. The debuggers "Hot Code Replacement" facility is a right step towards that direction.

  Posted by: rbirenheide on January 18, 2006 at 02:16 AM

• We don't even have the basics. The JDK doesn't even come with a "javai" to compile and run code. We have to javac and then java, and have to deal with intermediate files. Some IDEs don't even save and compile in the background.

  Only after that step does it makes sense to talk about changing code at runtime, whether through hot swap, class-loading hacks, serialisation or whatever.

  Posted by: tackline on January 18, 2006 at 05:53 AM

• Replace config files with source code....? Hmmmm....

  Customer: "Great app, but can I change the default directory where it saves its data?"
  Programmer: "Suuuuure. It's simple. All you need is to download a compiler, install an IDE, and read a few books on how to program Java."

  :-)

  Or what about...

  Customer: "That app you sold us has suddenly developed an error."
  Programmer: "Did you - ahem - 'configure' it?"
  Customer: "Sure."
  Programmer: "Right, well zip up the source code and mail it to me, I'll run it through a debugger and get back to you some time."

  :-)

  Posted by: javakiddy on January 18, 2006 at 07:04 AM

• Customer: "Great app, but can I change the default directory where it saves its data?"
  Programmer: "Suuuuure. It's simple. All you need is to download a compiler, install an IDE, and read a few books on how to program Java."

  :-)

  
  Or, you could provide a GUI page to do it. Which do you think the costumer actually wants? They don't care how it's stored, they just want to set the property. Most non-technical people would cringe at going thru a large config file and finding the line they need to edit and then edit it properly.

  
  Or what about...

  Customer: "That app you sold us has suddenly developed an error."
  Programmer: "Did you - ahem - 'configure' it?"
  Customer: "Sure."
  Programmer: "Right, well zip up the source code and mail it to me, I'll run it through a debugger and get back to you some time."

  :-)

  
  Are you seriously claiming that a customer can't somehow hose a system up with a poor config file just as bad? I don't buy it.

  Posted by: abickford on January 18, 2006 at 07:49 AM

• ok, ignore that all of text. i forgot how useless this thing is for posting and formatting.

  Posted by: abickford on January 18, 2006 at 07:51 AM

• great comments! the "customer: ..." examples made me laugh.

  on a more serious note, if you're trying to make sense of what i'm trying to say, just look at a typical ant build file and a typical rake build file. neither are meant for a customer to edit, but i contend that the rake file would be easier for the customer to understand and edit. finally, just because a java interpreter might process a config file does not mean that the config file can't constrain itself to doing only declarative things: x = 3, defaultdirectory = /some/path/

  anyhow, i hope this helps.

  Posted by: eitan on January 18, 2006 at 08:33 AM

• WebWork 2.2, released a few weeks ago, has a utility called QuickStart. It launches Jetty based on your project structure (rather than a packaged or exploded war) and compiles your .java sources directly for you. WHen you change a .java source, it recompiles it and keeps the application running, dynamically changing the class definition. Short of full support for HotSwap, this is the closest thing to "skipping the compile/deploy step".

  Posted by: plightbo on January 18, 2006 at 08:48 AM

• A customer really should never have to go to the config files. Maybe a system admin on a large scale deployment, but never the end user. So, that is just a software design issue more than anything. Many/Most programs simply aren't designed well, and nobody wants to admit it. If it's a simple contracted application which will not be installed outside of a single companies tech department then that is a different story...just use config files. If a end user must configure a system then provide the UI to do it.

  
  Interpreted languages will always be slower. Could always compile then run...if file changes - compile it - update the runtime. The other issue then becomes obfuscation and how to keep from making it easier for someone to highjack your code (anyone could do it then) or worse make it easier to plug a virus or some simple "delete as many files from the file system code"...obviously someone could use BCEL or something like that and do it now though....good to sign jars, so the runtime would have to checksum and sign the output as well on the user end.

  
  Why go to all that trouble just to keep from rebuilding your system? Most (nearly all) systems I've seen written in java don't take very long to compile...20 minutes tops...where as some C++ projects I've seen and worked on took hours to compile. So, really it's just people complaining about rebuilding software because maybe they're new to the game or technology they're using. Even if you have source files you still have to get changes and updates to the end user...so what is the big gain..you save 20 minutes??? If the design process were thought through better in all situations it would have a better outcome, and I think part of it deals with people thinking too much about how dynamic to make systems when a lot of them don't need to be that dynamic but just need to address their problem domain/use cases.

  
  Personally I think interpreted languages are really good for things like allowing end users to plug-in scripts to your object model and be able to manipulate your system for their needs if that fits the use case, but I think modules work well also, and really takes a combination of the two. I mean, the end user deployment is still going to have to be updated. So, there really is no gain on the end user side by having a pure interpreted language compared to bytecode....other than the byte code doesn't have to be created again, so that the code can run faster the first time it is run...sure could always make rebuilding part of the update process, but then doesn't that just push the time off onto the user? That was the whole point of bytecode...to speed up the runtime and help address the source code issues...yes at any point it is simply instructions...so anything can be reversed engineered and stolen, it just depends on how hard you can make it on the person doing it.

  Posted by: wadechandler on January 18, 2006 at 08:59 AM

• @eitan: I've actually just started doing this more often--was just today working on a chain to have a dynamic JMS publish and dynamic receive, with Beanshell on both ends. Once I worked out context issues (am I in a transaction? do I need a lookup?) it's wonderful to be able to edit code in jEdit, save, switch to jConsole (in jedit) and run the scripts. Nailgun is useful too, I'm using that in the middle, cuts down on BSH startup time (though I could host them in jEdit as well). Downsides are things like typos--don't get caught until the last moment, so if your script is doing something important, you have to add exception catches to make sure to clean up. But it's saving a lot of time.

  @javakiddy: mozilla and firefox configure themselves with javascript--they load a javascript file which calls back and initializes things. You can edit that JS or add to it yourself. it's a stable approach.

  Patrick

  Posted by: pdoubleya on January 18, 2006 at 09:04 AM

• Customer doesn't necessarily have to be an end user. For Ant, the 'customer' is the developer using it.

  @pdoubleya: I have no problem with interpreted languages being used inside applications - I've been using Rhino for years for this very purpose. The subtle, but important, difference here is that we are not talking about a scripting language being used from a compiled language, rather a compiled language being turned into a scripting one.

  The real problem here is that I configure BloggsBrowser and it requires me to know JavaScript. Then I configure AcmeMail and it wants be to know VB. Then I configure FreeOffice and I need to know Python. Then SuperFTP requires Perl, MegaMedia requires Ruby, CoolTunes requires Nifty...

  Then we all throw our hands up and cry "oh if only we'd stuck with XML. At least then when we got a new app we'd only have to learn what each config option did - not an entire new language as well... ;-)

  Posted by: javakiddy on January 18, 2006 at 09:46 AM

• Disagree. The idea of storing configuration data in Java source files is probably appealing for toy applications, but it is not suitable for sophisticated ones.

  First, configuration data do not have to be stored in files at all. Sophisticated applications have an SPI and allow to plug-in arbitrary configuration stores (LDAP for example).

  Second, if the configuration is stored in a file, then the file format should be both readable by humans and by machines (e.g., by graphical configuration editors). XML and Java property files meet both requirements. I believe, that Java sources are neither easily readable by "normal" humans, nor is it easy to write a configuration editor (compared to a simple property editor).

  Posted by: scotty69 on January 18, 2006 at 11:46 AM

• Having a separate compilation step is what allows us to keep Java a type-safe language, which is in my opinion what makes Java so strong of a language for writing production code. It's been shown over and over again that the earlier you can catch a bug the less expensive it is to fix, overall.

  If you don't care about type safety, then you don't need a language as verbose as Java. You're better off using scripting languages such as Groovy, which can be interpreted. If you haven't tried Groovy yet, give it a try - I think it does exactly what you're looking for. It keeps a syntax that looks a lot like Java (low learning curve), but it ditches the type safety and makes it less verbose and allows you to skip the compilation step altogether. Of course, there are many other scripting languages that do the same.

  Posted by: markroth8 on January 18, 2006 at 11:48 AM

• Mark Swanson mentioned BeanShell. I'll probably look at it before too long. But there was a ton of fuss about Groovy a year ago. Whatever happened to it? I remember something about it becoming a JSR then it's like it fell into a black hole and I don't hear anything about it anymore. Seems like one of these two interpreters or even Rhino might be a good solution.

  Eitan - sticking with Java, I'd love to see code/config samples from a couple ways of doing it "wrong" and a couple ways of doing it "right" according to your stated goal. And another question, do certain kinds of apps lead you toward one approach or another, or is there a global solution?

  Posted by: gerryg on January 18, 2006 at 12:02 PM

• I wonder what it would mean if .java files would suddenly be interpreted and executed when I double click on them, just like .vbs or .bat. I think that it would be better to open .java in editor by default (even though NetBeans does not support it), and instead have a separate Compile association, so that you could right click on .java, select Compile, which would create .class, and then double clicking on .class should execute it.

  So if regular JRE would have this new desktop integration, to Compile .java to .class, and to Execute .class (by default), then we would have it. Also in Windows we should have a wrapper to create .exe instead of .class, so that it could be executed in normal command line as well.

  As a side note, it is a bit odd that when I double click on .java on my machine, the file loads in Microsoft Development Environment. It has a good editor with syntax highlighting and all, but I would prefer NetBeans.

  Posted by: fuerte on January 19, 2006 at 07:25 AM

• I am all about going "back" to an interpreted approach to application development. I have to say that I have been a big fan of BeanShell (thank you Pat Niemeyer for developing it) and have been trying to incorporate more of it's dynamic/interpreted angle into my development.

  There are so many arguments for statically typed languages but I don't really see the need for these in _most_ cases. Generics and compile time checking can only do exactly what they were designed for: checking types. They cannot validate business logic! This is what we want. No compiler can tell you your code is correct, only that it is well-formed. So, these things basically become useless for really testing an application. So, we have developed the concept of test driven development. This is how we can easily drop the need for statically typing our code and just test it. While testing our business logic/algorithms we will also, as a wonderful side effect, be testing our types. This is all we need. The speed of development is increased due to immediate feedback of test cases, because we can run without having to do our whole recompile (and for a lot of us redeploy) and reduced lines of code. We can also do away with compicated build scripts (but that is just another side effect that I like). That's all I have to say about that.

  Oh, except for Groovy is still "on-track", but I am definately no fan-boy of that monster. I haven't a clue as to what they were thinking on that one. The syntax is so out there from the java space that it doesn't belong as the de-facto java scripting language. BeanShell is sweet. It's Java syntaxt, plain and simple (and I mean simple). You can drop types, implement and interface without declaring it and much much more. The only thing I want in it are true closures and simplified iterators like Ruby has. If we could get those wonderful code blocks it would be sweet.

  Anyway, have fun!
  

  Posted by: acidblue on January 19, 2006 at 04:57 PM

• 
  I'd love to see code/config samples from a couple ways of doing it "wrong" and a couple ways of doing it "right" according to your stated goal

  
  gerryg: you ask some very good questions. i realize i have done more to create more questions than to answer them. the first step should be awareness of the fact that, whether we like it or not, the issue of the compilation step has had a significant influence on our decision making, on the designs of our systems. i believe it has introduced a degree of complication that is unnecessary.

  i believe many comments point to solutions people have come up with in the realm of java: using beanshell, or rhino. although with rhino, we still remain in a heterogeneous environment: doing both java and javascript. these are entirely different languages. that was another big point in my original blog entry.

  i also look for examples in "other worlds" such as smalltalk and ruby. for many years i was blissfully ignorant of other languages. i am in awe of smalltalk at the moment. everything is interpreted. so many things are in-band, completely transparent, because everything is written in smalltalk: if statements, exception handling are all implemented in the language; they do not require the designers of the language to give us a construct for it. but i digress.

  i really like what acidblue had to say. two more points: [a] even though i might have mentioned config "files", i did not imply any location dependence. i don't care if the stuff comes out of a database or jndi. [b] although most of us 'got it', allow me to repeat: just because a config file can be interpreted by java or beanshell (or whatever) does not mean that it can't remain as simple as a properties file: completely declarative. the upside is that if at a later point in time, the config file needs to become more complex (as it inevitably does), then we don't have to suddenly build complex extensions to the mini-language we decided to invent.
  

  Posted by: eitan on January 20, 2006 at 11:30 AM

This work is licensed under a Creative Commons License.