Bruce Chapman's Blog

Dear Santa - Exceptional Stuff

Dear Santa,

My stack traces seem to have been getting less and less useful recently. Just to make myself perfectly clear, I am not talking about wrapping the sleigh reins around a chimney, which is what you might think of when you hear the words "stack trace".

So here are a couple of things I'd like your seven elves to work on please.

Better messages in JVM thrown Exceptions

First, can the JVM elves please use the information that is available when the JVM throws an exception to create a useful message. For instance, in cases of throwing a NullPointerException, could we please be told the type of the reference that is null, and what it is that can't be done to null.

For example, say you call List.get() on a null reference, wouldn't it be nice to see something like this in the message

Exception in thread "main" java.lang.NullPointerException: ((java.util.List)null).get()

Exception in thread "main" java.lang.NullPointerException: ((String[])null)[]

As best I can tell this can all be done without actually changing any specs. There are a few related bugs and RFEs in bug parade that address little parts of this, but what I'd really like is for someone to go through the whole JVM Spec, look at each exception that can be thrown by each opcode, and put all the useful information (types and names) available to the JVM into the messages in some consistent notation.

Context Identifiers in StackTraceElements

This suggestion might be a little more controversial. Often the code that is being debugged to find the cause of an Exception is not java code, but some other form that has been converted into a composite data structure and is being "executed" (in loose terms) by java code that someone else wrote for an external library or framework.

Some examples are JSP or JSF pages and scripting language code being run by an interpreter.

What would be really helpful in these cases would be for the stack trace to include a small amount of contextual information associated with key objects in the stack. Wouldn't it be nice if you could see the JSP / JSF EL expression that was being evaluated when an Exception was thrown. Wouldn't it be nice to see the name and source line number of the XML elements that enclosed the element containing that expression? Wouldn't it be nice if you could see the line numbers of the javascript code that was being executed? Wouldn't it be nice if your own classes could add the contents of a String instance field to a stack trace element for specified methods of key classes.

Here's how it might work.

Add a new property to StackTraceElement to hold the context identifier.

Define a marker annotation (say @StackContext).

Use @StackContext to annotate the methods for which you would like the context identifier shown in the stack trace.

For any class with any methods annotated with @StackContext, annotate a single String field with @StackContext to identify the context value.

When a stack trace is being filled in, the JVM would look at each method on the stack, and if it was annotated with @StackContext then the @StackContext field would be accessed, and if non null, then the value of that field would be used to set the context identifier in the corresponding stack trace element.

Printing the stack trace would print the instance identifiers where they were non null.

Other details might include taking a prefix substring if the annotated field's value was too long (say no more than 40 characters), and only filling in the context identifier once for consecutive stack trace elements with the same "this".

Imagine.

Comments

• Hi Bruce,

  In fact JXInsight's Diagnostics runtime and programming interface are extremely lightweight. This is because as you note above only particular methods are of interest, our runtime is finely tuned offering fast load-time weaving byte-code instrumentation, and the data (runtime state: args, target, exception) is only pulled when an exception is thrown and diagnostics dumps has been enabled for the JVM.

  It is important to note that at anytime the contextual diagnostic stack can be viewed by taking an snapshot from within our console and not just when an exception occurs. Making this a lightweight runtime state and request flow debugger for production environments. We also make a distinction between the class of the code executing and the target class the method is being dispatched on which is something a stack trace cannot do. Oh before I forget a diagnostic image dump can also include stack frames from other threads - very important when work involves more than testing Java desktop applications!

  regards,
  William
  

  Posted by: wlouth on August 17, 2007 at 09:48 AM

• All: Thanks for the feedback

  William: Interesting stuff, I can see the value in that for specific debugging situations. I was imagining a mechanism more lightweight but ubiquitous, I think there is value in both.

  James: The first of those RFEs is probably the one that most accurately reflects the first idea, but I'd extend it to all JVM generated exceptions except the Errors (like OutOfMemoryError ) which are pretty much unrelated to the actual context in which they ocur.

  jlrowe: Most of the JDK is reasonable in terms of context information at the point of failure. The weaknesses at present seem to be the JVM thrown exceptions (especially NPE), and where the important context (to the reader of the stack trace) is at a different level within the stack and so is unrelated to and not even available to the level that constructs the Exception.

  ksenj: It was not my intention to suggest that the @StackContext would apply to all (or even most) classes, nor to every method within some classes. In fact in a production library it would only be relevant to a few key methods of a few key classes. I looked through a (deliberate) stack trace in our JSF app, and of the approx 60 methods on the stack, there was one that would be a definite candidate, about 4 more where the context data might be helpful, plus a couple of others which were worth some consideration.

  I am not a JVM expert by any means but I imagine in terms of performance, the JVM could augment its class data at class loader time, so that the annotated methods had a bit set to say so, and the class itself could have a offset or pointer to speed up finding the @StackContext field so that it didn't need to look at every field and look at all its annotations to see if it was THE ONE. Stack trace fill in would need to look at the bit on the method and if set (usually not) find the class (it has to do that already to fill in the other parts of stack trace), reference the @StackContext field using the shortcut above, and copy its value to the stack trace element, no big deal. Trimming the string could be done IFF the context info was accessed rather than when the stack trace was filled in. Another consideration is memory "leaks" but Strings can't hold references to other objects so although the Exception would stop the String itself from being garbage collected till the exception was dereferenced, I can't see how that would cause any problems, even if you stored the caught exception at some point, It'd only be the Strings, not any other parts of the "this" objects at each level of the stack. I can't be sure, but I suspect this could be done in such a way that the overhead was minimal.
  

  Posted by: brucechapman on August 17, 2007 at 01:34 AM

• 
  "Wouldn't adding context information on each and every method make creating the exception even more expensive especially when you have a huge stack trace (very typical of web applications to have huge stack traces starting from filters, servlet, mvc, middle tier, database tier etc)"

  
  Yes that is why most commercial and production oriented diagnostic tools such as JXInsight offer the ability to filter which exceptions result in the generation of diagnostic images and which particular methods have additional context. I think it is always important to separate the code call stack from the contextual trace / diagnostic stack at least at runtime.

  regards,

  William
  

  Posted by: wlouth on August 16, 2007 at 09:37 PM

• Wouldn't adding context information on each and every method make creating the exception even more expensive especially when you have a huge stack trace (very typical of web applications to have huge stack traces starting from filters, servlet, mvc, middle tier, database tier etc)

  Posted by: ksenji on August 16, 2007 at 04:01 PM

• No context in exceptions is one of my personal bugbears, if you have some context when you throw the exception why not just add it in to the message string? I do this in all of my code and it definitely improves serviceability.

  Going through the JDK and adding context to the exceptions would set a great example, and definitely make life easier.

  Posted by: jlrowe on August 16, 2007 at 02:34 AM

• While we are at it, why not more information in the Exceptions themselves. For example: FileNotFoundException has no method getFileName() to get the file which does not exist. For more details see JavaExceptionsAreFirstClassObjects .
  

  Posted by: vhi on August 15, 2007 at 11:25 AM

• I have entered two RFEs that are very simliar to your requests:
  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4834738
  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4857236
  Have you entered RFEs for your requests?

  Posted by: staufferjames on August 15, 2007 at 06:42 AM

• Hi Bruce,

  I recognized this problem early on when we creating our profiler and not having access to a debugger because of the inability to load two agents into
  http://www.jinspired.com/products/jxinsight/api/com/jinspired/jxinsight/diagnostics/Diagnostics.html

  Here is how runtime state inspection looks like within a production software tool which is an important aspect of any design considered.
  JXInsight Diagnostics

  Other articles related to diagnostics are available here

  We are also making this much easier within our AspectJ aspect library extensions which will also be supported by a similar annotation API as listed above which by the way we designed when we built a custom integration (using XDoclet + AspectJ) for the biggest IT company - way back in 2003. At that time your StackContext was a Trace JavaDoc tag.

  JXInsight 5.5 and AspectJ - Sample Application

  I would really like to see our 3 APIs (Traces, Probes, and Diagnostics) become part of the runtime API instead of seeing java.util.logging abused over and over again.
  regards,

  William Louth
  JXInsight Product Architect
  JINSPIRED

  Posted by: wlouth on August 14, 2007 at 11:03 PM

This work is licensed under a Creative Commons License.