In response to my A Read-only database in a jar? one of the commenters remarked that "Derby is alright but is much slower than HSQL" and referred to a benchmark at http://jamie.ideasasylum.com/notebook/index.php?id=4.

And guess what? He's is right. The problem is to figure out how he's right. I had one of our performance engineers look at the benchmark source and he came back telling me that this benchmark basically measures how fast the application connects to the databases and how fast they compile SQL.

And that's great. As long as your concern is connect times and SQL compilation. But if this benchmak was rewritten to be more economic with connects (e.g. with connection pooling) and to use prepared statements more systematically, the numbers might change dramatically.

It's easy to refer to a benchmark, but it's not always easy to communicate what that really means. The statement about "Derby is much slower than HSQL" is of limited value to you outside the context of

1. what the referred benchmark measures and
2. what your requirements are.

If you go to TPC and read about the various benchmarks (and these benchmarks are the real serious stuff, not some homegrown performance tests called "benchmarks" and put on the web), you'll soon realize that there are no such thing as a slower or faster database. They are only slower or faster given a specific set of criteria in a given benchmark.

In the end, a database that some benchmark labels as slow may just be what you need. If it performs fast enough and have the right functionality. Similarly, I don't buy the fastest car in the market. I don't even buy the fastest car I can afford. I buy a car I can afford that fits my needs... e.g. drive to the northern Norway with 2 grown-ups, 3 kids, a kayak, sleeping bags, a large tent, glacier climbing equipment, food, clothes and so on. Can't do that with a Lamborghini.