Fabrizio Giudici's Blog

Historic series of profiling data

After fighting with some showstoppers in the NetBeans Profiler (involving RCP projects) and finding a decent workaround, I've started the tuning of the Metadata facility for blueMarine. I've already done tuning in the past, of course, but I have always had some frustration in how I easily lost the traceability of the thing. I'm giving an example: a few days a go I ran the profiler and got some figures; in particular two methods were the hot spots of the test and I realized it was not good. Working on those methods, I was able to push them down in the list. To help during the job, I exported some dumps of the NetBeans Profiler, manually spotted the most important numbers and manually tracked them build after build.

Now the hot spots distribution is better, but there's still work to do and I won't be able to work on it before some days. When I'll put some further optimization, I will get a new list of hot spots that will look better. And so on until I feel satisfied for this interaction. Then I'll work for several weeks again on new features and bug fixing, leaving tuning alone. I'll run another tuning session probably in a couple of months, etc. By that time, it could happen that the design changes have at least partially invalidated the optimizations done in the current tuning session. Looking back at the code I wrote years ago (and some code I wrote for customers) I can see a lot of maybe-smart optimizations that now could be pretty useless, since the context where they were good has changed (often I see in customers code that this is even made worse by premature optimization, which greatly increases the chances that the selected optimizations are inappropriate).

Of course, you can deal with it: just run another session of profiling, get the new list of hot spots, etc... The point is that since I run profiling sessions every in a while, I'm likely to have forgotten a bit about the context. I can fix this by writing comments and entries in the issue tracker, but this is time expensive. My point is that it would be nice to put some automatic stuff in the CI facility. For instance, I could run a test in profiling mode, dump the hot spots data and plot a graph showing the values (percentage and time ticks) for the five methods on the top of the list and a sixth record with the data of all the rest. This would save me a lot of time and - in the spirit of CI - it would be done incrementally for each build, allowing you to relate repository modifications with sudden changes in the performance (e.g. you see that a method at a certain build has suddenly moved up to the top of the hot spots, and you could easily track the commit version and find out which changes are responsible for that.

What do you think? I don't think there's a plugin for Hudson already available, but I could make one - I think there's already a plugin for plotting graphs if you produce a file with a given format, the only thing I need is a (FLOSS) library that allows me to read a profiler dump.

Comments

• BTW, the NetBeans engineering just confirmed that I could use a part of the NetBeans Profiler APIs. I'll look into that.

  Posted by: fabriziogiudici on April 08, 2008 at 03:22 AM

• Hi Fabrizio
  
  You could always use JXInsight's Probe instrumentation (via AspectJ) and then access the generated metering via the same Open API that is used by the instrumentation as well as our server in generating a snapshot from within the same JVM. No need to investigate a way to inspection the nb profiler dump format. You would get more data (not just the clock time) and at a lower overhead.
  
  http://www.jinspired.com/products/jxinsight/api/
  Here is a blog entry showing how to use our production performance management solution as a code profiler.
  http://blog.jinspired.com/?p=223
  William

  Posted by: wlouth on April 09, 2008 at 12:17 AM

This work is licensed under a Creative Commons License.