Will Iverson's Blog

The Future of Native Interfaces

I got a lot of questions about JNI after my Mac OS X for Java Geeks book was published. I just wrote an article for ONJava, Mac OS X JNI Revisited. While writing the article, I was mostly focused on answering reader questions, but as I was working, I started to wonder why JNI development was just so hard.

When I was working at Apple, there was some interesting work done on native integration called JDirect (I believe that Patrick Beard was the one who came up with the idea). It's been a while (five? six years?) but my recollection was that basically the JVM was smartened up enough to call native libraries without the JNI cruft - especially nice for using existing native system libraries (I believe mostly CFM) for things like QuickTime integration.

I seem to recall more than one complaint that it was "too easy" to integrate native functionality.

The latest JDK 1.4.1 release from Apple removed JDirect functionality after being deprecated for quite some time. One of the big problems with JDirect was that it was Mac OS only, and while there was at least some notion of Microsoft including a similar technology in their JVM, it obviously never got rolled back into the JDK proper.

Probably the biggest problem with the current javah-based package is the tremendous difficulty using existing native libraries - there's no reverse version of javah that sucks up existing header files and produces a set of Java classes filled with native keywords that automatically binds.

I'll be blunt - I don't know if that's the sort of thing that ought to be included in the JDK or not. Maybe it's a feature for a C/C++ linker - to automatically generate the bindings as a side product if fed a particular flag. Or, maybe this is something best handled by xdoclet/ant/maven, at least automating some of the drudgery of adding new signatures to your C/C++ library as you add new native code?

So, I guess the question really is, is there a place for improving these native integration tools? Should it be done in the JVM itself, in the JDK tools, or as an external project? Or is there even enough interest in Java-native bridging for it to matter - it's possible now, there's a bit of a steep learning curve, and that's just ok?

Oh, and as a final note - somebody asked about REALbasic, which of course bring up the whole concept of bridging non-C/C++ code with Java. Visions of SOM, DSOM, and CORBA leap to mind... someday the nightmares will stop...

Comments

• reverse javah
  "Probably the biggest problem with the current javah-based package is the tremendous difficulty using existing native libraries - there's no reverse version of javah that sucks up existing header files and produces a set of Java classes filled with native keywords that automatically binds."

  Actually some work has been done in that area. Check out the JOGL project here and you will see that the native binds are generated automatically. I believe the tool used is 'antlr'.

  Posted by: swpalmer on October 09, 2003 at 12:09 PM

• Have you seen SWIG?
  SWIG (www.swig.org) might do what you need. It claims to support Java.

  Posted by: jimweirich on October 10, 2003 at 06:42 AM

• The GNU Compiler for Java has CNI
  If you build with gcj there is a JNI alternative called CNI.
  

  Posted by: ahornby on October 10, 2003 at 08:39 AM

• noodle glue
  Hi -
  For the last year or so we've been working on a project which integrates Java and C++ pretty transparently. It's possible, for instance, to subclass a C++ class in Java and vice-versa. Java objects can be passed to C++ code and vice-versa. Garbage collection of C++ objects is also (kind of) handled (in the current version you have to be careful about circular references which go from C++ into Java and back to C++ again).

  All of the JNI code generation is automatic and is generated by a couple of programs which automatically parse your existing C++ header files (or java files, though we haven't quite finished that bit yet) and generate corresponding classes and bridging code in the other language.

  Although it's possible to work with any Vanilla C++ object, normally you subclass your C++ classes from our base class, which does reference counting and a few other housekeeping tasks. If you don't subclass it the system creates another object to manage your plain C++ object.

  We bridge STL template classes, like Strings etc. more or less transparently. it's also possible (and fairly easy) to create custom "translators" to optimise the bridging of a particular class.

  It's also possible to add XDoclet style tags to your source code to provide some extra hints for code generation, to do things like generate Java BeanInfos etc.

  We have built the system for use building our own project, but are thinking of making it open source if there is enough interest. But at the moment we're a bit wary that it will be an enormous amount of work creating a decent website and documentation and supporting the first users, and we are currently working to a very tight deadline to finish our current project.

  i'd like to gauge how much interest is out there in this kind of thing before we do a lot more work "productising" it. Does anyone think it would be useful enough that they would be prepared to help doing all the work necessary to support a decent open source release ?

  Posted by: jportway on October 11, 2003 at 02:53 PM

• RE: noodle glue
  Wow.... that's a lot of stuff. Couple of questions...

  - What platforms have you tried it on?
  - What sort of headers have you tried this on? Any lumps of code by name (just to suss out how reliable?)
  - Is the code gen regeneratable without destruction? i.e. let's say you add a method to a header, will it then add that new method to the C code without blowing away your existing .c/.cpp?

  I'd be happy to take a look through it and give you my two cents on how much work/effort/etc. would be involved to get it pulled together into a "real" open source project... feel free to email me at...

  wiverson AT cascadetg.com

  ...and I'll let you know what I think.
  

  Posted by: wiverson on October 11, 2003 at 04:55 PM

• RE: noodle glue
  We currently compile on Mac OSX and Windows (visual studio).

  To give an idea of the capabilities, have a look at openSceneGraph (http://www.openscenegraph.org). It's a pretty big bit of code which uses a lot of templates and c++ tricks, but it quite clean -we currently bridge around 99% of it. One of the things that makes OSG good for us is that it has a reference counted base class, which most of it's public class inherit from, so we just mix our base class into that. As i said before it's possible to bridge "plain old C++ classes", just a bit more fiddly.

  The source code isn't regeneratable as such - but it's not an issue because of the way Noodle Glue works. Noodle Glue doesn't generate empty stub methods like javah. You write your C++ (or java) code normally, then you run our stuff on it, and it produces identical classes in the other language. C++ abstract classes turn into java interfaces, enumerations turn into java typesafe enumeration classes.

  For instance, if you have an existing c++ class called "foo.cpp", Noodle glue will produce some JNI C files and a "foo.java" class which will have exactly the same methods as the c++ class. If you want to add functionality to it, then you can just subclass it in java, and override any methods you want to change, just as if it was a Java class. If you want you can then pass an instance of this new java subclass back into C++ and if the overridden methods are called then the java code will be executed. Basically you can simply treat the bridged classes as if they were pure Java classes.

  We're very, very busy for the next couple of weeks, working to finish a deadline - after that we will try to get something presentable to release. If you'd like to see the java version of open scene graph send me an email at josh (at) stain.org and I'll try to get it to you.
  

  Posted by: jportway on October 12, 2003 at 12:22 PM

• mixed source code
  Yes, there is (a lot of) place for impovement.

  Think about the ability to mix Java and C/C++ in the same source file. Or Java & Python. Or ...

  To get an ideea take a look at Perl's Inline module.

  Posted by: razvanm on October 13, 2003 at 05:23 AM

• RE: noodle glue
  I know I'd have paid good money for a tool like that a couple of years ago. As I still have to wrestle with the nightmare known as JNI, I'd STILL pay money for it.

  I suspect I'm not alone.
  

  Posted by: commonlook on October 14, 2003 at 06:50 AM

• RE: noodle glue
  Well, when we get time to clean NoodleGlue up you won't have to. We plan to make it GPL so it'll be free. However, it's unlikely to be available for at least a couple of months as there are a few things that need to be added and fixed to make the process user friendly.

  Compared to what else is out there I think it compares very favorably. We have looked at most of the options offered on this page on and off over the last couple of years and over all we still think NoodleGlue is easier to use and support more features than all of the alternatives.
  We looked at SWIG nearly 2 years ago when we first started our main project. The SWIG java bindings just weren't up to the job we required of them. I had a look recently and it has certainly got much more complete, but it still appeared to require a load of interface files or header file modification. From my cursory look I think NoodleGlue handled more of C++ and with less effort for the coder. You can add tags in your header comments or in an XML script, but there is rarely ever any need for this.
  gcj is a great idea but isn't much use for code you want to just wrap up and use it. There are lots of OpenSource libs which have been translated for gcj though and it has great features like built-in garbage collection and the ability to compile java down to native code. In practice though it is more like a hybrid language, and not portable to other compilers. NoodleGlue has a garbage collection mechanism for reference-counted classes, and is about as safe as you can get for bridging standard unreference counted C and C++. And (in theory) it works on most compilers.
  I had a quick look at gluegen utility which JOGL use. It does what they need, but again is missing numererous C++ features NoodleGlue supports.
  The most comprehensive wrapping utility I have seen is actually not for Java but for Python and is available from boost.org. I think NoodleGlue does supports about the same range of C and C++ features as the Boost Python generator. I think our inheritance is probably more flexible though.
  Well anyone who is interested will be able to see for themselves when we have time to get a distrubution sorted out.

  Posted by: tree on October 14, 2003 at 05:48 PM

• RE: noodle glue
  Sounds pretty cool with a lot of features. It reminds me of the JACE utility (http://sourceforge.net/projects/jace/). Have you looked at their stuff?

  Alan Thompson

  Posted by: catalan on October 16, 2003 at 01:06 PM

• RE: noodle glue
  Yeah I had forgotten about JACE. I think we looked at it, although a long time ago. JACE as far as I can tell only works one way so it generates C++ classes based on Java classes. It looks like is quite nice, the code is quite like gcj without the compiler dependency.
  NoodleGlue generates code both ways, although I suspect JACE is better at the making C++ proxies for Java classes just now, as we have neglected this direction for quite some time, as we never use it.

  Posted by: tree on October 17, 2003 at 11:03 AM

• RE: noodle glue
  A little late, but I thought I'd mention that Jace does go "both ways". You use Proxy classes to call Java from C++ and Peer classes to call C++ from Java.

  God bless,
  -Toby Reyelts

  Posted by: rreyelts on October 31, 2003 at 11:21 AM

• J++, Jawin
  Yeah, J++ had something called J/Direct too, which was quite easy to use. .Net has something similar... I agree, it should definitely be made easier, native integration is one of the few major areas where .net has a significant advantage.

  Interestingly, the jawin project on sourceforge gives your java code the ability to call any windows DLL function (could be any windows DLL, not just a microsoft one) without writing any JNI or native code... Something I might have considered if I hadn't already written all of mine in JNI...

  Posted by: kenlars99 on December 23, 2003 at 03:33 PM

• wow power leveling
  wow powerleveling
  wow power leveling
  wow gold
  wow items
  feelingame.com
  wow tips
  Most Valuable WOW Power Leveling Service
  wow power leveling faq
  cheap wow power leveling
  wow power leveling
  wow powerleveling
  wow power lvl

  Posted by: wowleveling3 on December 13, 2007 at 06:05 PM

• 网络营销软件
  网络营销软件
  网络营销软件
  群发软件
  群发软件
  ---
  群发软件
  网络营销软件
  论坛群发软件
  网站排名软件
  群发软件
  推广小助手破解版
  论坛群发软件
  网站排名软件
  群发软件
  网络营销软件
  网站推广软件
  信息群发软件
  论坛群发软件
  信息群发软件
  博客群发软件
  qq群发软件
  邮件群发软件
  博客群建软件
  企业名录搜索软件
  信息群发软件
  邮件群发软件
  论坛群发软件
  博客群发软件
  网站推广软件
  网络营销软件
  全能营销破解版

  Posted by: uuok999 on December 20, 2007 at 12:24 AM

This work is licensed under a Creative Commons License.