John Reynolds's Blog

Why Use A Database Instead Of ____?

A recent blog entry by Simon Morris questions the usefulness of including a relational database (JavaDB) with the Standard Edition of the Java Development Kit (SE JDK). The JDK is primarily intended for the development of Desktop applications, and in Simon's words:

"most real desktop applications (browsers, players, word processors, video editors) are not database heavy, why is Java DB being included in the SE JDK?"

Let's take look at word processors: A very common business function for a word processor is to create a "form letter" and populate fields of the letter with data that comes from another source (sometimes a database, sometimes a flat file of some sort). Of course the word processor's author could come up with a custom mechanism for extracting data fields from a file... But if an RDBMS (Relational Database Management System) is used, much less custom code is required, and (much more importantly) the data will be stored in a format that makes it available to other applications.

Okay... So maybe you agree with me that storing data in a format that is accessible to many applications is a good idea, but why not use XML (eXtensible Markup Language) files instead of an RDBMS. XML files are very easy to parse, and they are very easy to extend over time.

True... But what if multiple applications need to be creating, updating and deleting data simultaneously?

All "real" Database Management Systems are designed from the ground up to handle simultaneous inserts, updates, and deletions. Options abound for managing simultaneous access to tables and records, and for handling collisions and potential deadlocks. Many people have worked for many years to fine tune these systems, and they work very, very well.

Okay... So maybe you are beginning to agree that an RDBMS is a good idea for inclusion in any development kit... but why JavaDB? It's not a "real" RDBMS. It's just a toy, and there are many, many alternatives.

Now we have reached an agreement of sorts. I don't really understand the merits of JavaDB over HSQLDB or any other contender... but I also have to admit that I really don't care all that much.

What is important to me is that an RDBMS is included (by default) in the development kit. Inclusion of something like JavaDB means that every single Java tutorial can assume the presence of an RDBMS. Every new programmer learning Java can painlessly learn to program using JDBC (Java Database Connectivity) and learn enough about SQL to keep them from being dangerous.

But wait... Isn't Java Object Oriented? Shouldn't Java developers be focused on using persistent objects rather than fields from a database?

I know that it's contrary to promote tables and fields over persistent objects, but in defense of this position I'd like to share with you some insights that Ken Orr (of Warnier-Orr diagram fame) shared with me (highly paraphrased below)...

"An object is just a single hierarchical view of the underlying data. With an RDBMS, you can support many views of the data... each of which is appropriate for the application that needs it. Constraining data to a single view (as an object) is robbing the data of some of its worth"

Data is often a company's true "treasure"... They slice it, they dice it, they analyze it right, left, up, and down. Reports are written against this data. Presentations are built from this data. Businesses literally live or die on their ability to leverage this data.

This is true whether you are writing a Browser-based application for a business, or whether you are writing a Desktop application for a business. Data is really important, and an RDBMS is a really good tool for handling data.

Comments

• I recently wrote an article at work where I speculated on the waning usefulness of RDBMS systems. The kicker is one of the many google-effects: if searching is more difficult than querying google, forget it. I.e. RDBMS force you to do structured search and I don't think that will cut it for the new generation of users.
  Add to this the fact that 80% of your real-world data is unstructured, or only partially structured, the third normal form doesn't fit well.
  Enter XML databases which allow you to do an unstructured query, but also a structured query when you need to. You can choose which nodes you wish returned, you don't need to get a whole hierarchy back. You don't even need to have fixed schemas.
  Some random thoughts:

  In one of your comments in Simon's blog you mention the "RDBMS is the root of all" mindset that you wish to instill into new developers. While I recognise that an RDBMS may be a useful tool, I have always argued against the mindset that wishes to put an RDBMS into everything. (as I have against any mindset that presupposes a certain solution to be necessary).
  Including JavaDB in the sdk will impede competition. JavaDB seems to be complete and stable (HSQLDB is not worth mentioning in the same breath), but it also seems to be fairly slow so development is nowhere near finished in the pure java db space.
  I thought that "SOA-think" was the new thing, not "DB-think"...

  
  So why bloat the jdk?

  Posted by: tobega on July 14, 2006 at 12:59 AM

• 80% of the data is unstructured, perhaps. But at the same time 80% of the applications rely on structured data.

  Why a DBMS must be a relational one I don't know (besides the fact that nearly all enterprise-strength DBs are realtional), the key point is that accessing DBMS is well understood technique used by the majority of the applications.

  The JavaDB just gives easy access to a JDBC-4 accessible DB for all developers, even those who think that all they need can be found on the web (although of most of it is merely accessible via the web).

  XML without a schema is a nested list. Period. Most case I encounter some "flexible, un-constrainted XML" it is just replacing thinking by runtime-errors and/or data-loss.

  SOA. Some people say that any problem can be solved just by adding an other layer of indirections. Problem with that: Information is invariant under indirection, the data entered has to go somewhere and the requested data has to come from some storage.
  

  Posted by: csar on July 14, 2006 at 01:45 AM

• Tobega,Csar's replies to your comment are pretty much what I would have said (thanks csar).
  As for JDK bloat... I agree that including more and more features in a development kit could get out of hand... but I feel pretty strongly that since SQL is the underpinning for a whole generation of applications, it would be a mistake to continue to leave it out.On a similar note, I fully support including WS client and XML capabilities in the JDK, since that's very likely the basis for many client applications in the current generation of applications.As for the JRE... That's where we really need to worry about bloat. I'd love to see a more modularized JRE.-JohnR

  Posted by: johnreynolds on July 14, 2006 at 05:58 AM

• Well, I suppose you will find the coming years painful then. I am sure you are going to need ways to deal with semi-structured data. In fact, the user requirements for basically every project I've been on have called for semi-structured data but so far we have never been able to provide it because it didn't fit in our database model.

  Posted by: tobega on July 14, 2006 at 09:00 AM

• Tobega,
  Of course we'll need ways to deal with semi-structured data... And we will need ways to deal with simultaneous access to that semi-structured data... Perhaps true Object/Relational DBMS systems will be part of the answer (I worked on a pretty good one called UniSQL back in the 90's, but it never caught on).

  Except in the trivial case where a single user application is working on a private data set, data access is not really an infrastructure service where you want to "roll your own solution"... That's why I advocate including well thought out data access mechanisms (like JDBC and a default RDBMS) in the JDK.

  -JohnR

  Posted by: johnreynolds on July 14, 2006 at 09:38 AM

• I'm not opposed to having the JavaDB available in the JDK, but something you just said got me thinking, John.
  The JavaDB is available to me as the developer, but what about my end-users? If it's not part of the JRE then I need to make sure there's a JavaDB server available somewhere for them.
  Even though I'm using JDBC, should I use JavaDB for development if my end-users will be using Microsoft SQL or MySQL?
  Csar says that the JavaDB "...gives easy access to a JDBC-4 accessible DB for all developers", but if I need to download and install JavaDB for the end-users, shouldn't I download and install it separately myself so I know what to expect?
  I suppose it's akin to having Swing in the JDK...I don't use Swing all the time but it's there for when I do use it, but then, Swing is available right away to my end-users.
  The idea of a word processor with RDBMS is intriguing, but do I assume a dedicated JavaDB for the word processor? I know it's going to happen on the JDK side, but on the JRE side...will there be multiple instances of the JavaDB installed that need to be considered?
  I'm just rambling some of my thoughts...I'm not familiar with the JavaDB system...feel free to expand on them.
  DB

  Posted by: dblair on July 14, 2006 at 12:58 PM

• DB,If I had "my way"... then downloading the JavaDB would be part of a much more modularized JRE, where necessary components are automatically downloaded when needed by an application.
  -JohnR

  Posted by: johnreynolds on July 14, 2006 at 04:56 PM

• 
  The Java DB API is dead weight to 99% of all applications that use the SDK & JRE. Let's not kid ourselves - it is a fact 99% of applications will never use Java DB. Sure, more than 1% of applications use a database, but that database won't be Java DB.

  Interesting that more than 1% of all Java applications are J2EE, and more than 1% of all Java applications are web services. But neither technology distribution (J2EE or the SJWSDP) made it into the core SDK APIs - you must download them separately.

  So, why include a RDBMS in the SDK or API? Why include an API in the core product that less than 1% of the Java community will use? Does Sun hope Java to become Microsoft Access or Foxpro by database inclusion?
  

  Posted by: phlogistic on July 16, 2006 at 05:25 AM

• Phlogic,I agree with you that the JavaDB probably won't be the database of choice for more than a tiny percentage production systems.
  Your comment about Web Services and J2EE applications is a really good observation, but I would hasten to add that most Web Services and J2EE applications rely on an RDBMS, and I'd probably wager that JDBC plays a role in most of those. I'd guess that most developers need to have a JDBC compliant RDBMS installed on their development machines.All of the comments that have been made on this topic have helped me better understand why I feel that inclusion of an RDBMS is important in the base JDK...It boils down to this question: "What core technologies should all Java programmers know?"

  My experiences have led me to believe that all Java programmers should have a firm understanding of SQL... and know how to effectively use JDBC. Many will use O/R technologies... but the more they understand the underlying "R", the better off they will be.From that perspective, including a default/stub RDBMS with the JDK seems like a natural thing to do... and that's why I was glad to hear of Sun's decision.

  My guess is that much of the negative response to Sun's decision has more to do with being reminded that Sun is free to make any decisions that is wants to with respect to what is (and what isn't) included in the JDK.

  -JohnR

  Posted by: johnreynolds on July 16, 2006 at 08:45 AM

• 
  That's a good rationale for putting items in the core JRE / SDK distribution (What core technologies should all Java programmers know?)

  But then I wonder - why not put EJB3, J2EE, & web services in the core distribution, then? All Java programmers should know most web technologies. And remove Swing / AWT, printing, and sound?

  Yes in the end I want Sun to control Java, rather than an open source model or something similiar to the W3C or (W.ait 3. C.enturies for a change). Just having a problem understanding the "why" of inclusive JavaDB.
  

  Posted by: phlogistic on July 17, 2006 at 09:33 AM

• phlogistic, not all Java developers are web programmers. Lots of people (myself) included work on systems that have a rich client (J2SE/Swing) and J2EE server, with no web stuff whatsoever.

  Essentially, Sun needs to decide what J2SE is: is it desktop Java (which is how we used to think of it)? Is it the "base" version of Java that should include things useful to everyone? Is it something else entirely, such as a base for lightweight POJO-based apps?

  I really don't think that the three-way split of EE, SE, and ME works anymore. Desktop developers certainly aren't thrilled with Java, and everyone else is annoyed at the inclusion of Swing, AWT, etc. (witness the above post). Maybe Sun really needs to do a Desktop Edition (or better yet, extensions) or something to sort this whole thing out.
  

  Posted by: rlinwood on July 17, 2006 at 01:58 PM

• To me, the real problem is JavaDB. Why not Derby? I know they are the same today, but they will inevitably diverge.
  The same is happening with Rhino. The Rhino distributed with Mustang is not the most current Rhino.
  Do you enjoy dealing with different versions of the same tool? Are you looking forward to extending the lifetime of outdated versions? Do you think that maintaining these separate versions is a good use of Sun's resources?
  If something is really important and should be maintained forever, go ahead and put it into the JRE. One can make an argument that a lightweight SQL database belongs into every SE client. Or not. If it isn't important enough for the JRE, then don't put it in the JDK.
  If Sun must coddle those developers that can't handle the extra effort of downloading Derby and Rhino, just make another bundle, like the JDK+Netbeans buundle.

  Posted by: cayhorstmann on July 18, 2006 at 07:27 AM

• Good points on Derby and Rhino... and I personally like bundles.As for the JRE... I really would love to see it modularized at a much more granular level.-JohnR

  Posted by: johnreynolds on July 18, 2006 at 07:53 AM

• Part of this discussion reminds me of an old colleage in Copenhagen who said that he didn't find the need for a database at all. If he wanted one, he would write one! And the DBMSs in question then (1990s) were real mainframe iron databases! The original post here at least convinced us the need for a database. Now Javadb. Do we need a Javadb? Well, a better question would have been : Do we have a real DBMS here? 'Database' is a misused term! In the 1980s dBase was termed as a database by many. Wikipædia even now harmlessly but explicitly says'dBase was the first widely used database management system (DBMS) for microcomputers'.

  Was dBase DBMS? Not at all. dBase was only a filesystem which allowed database-like access on the PC. dBase tables were readable using other software, because all you need to do was to decode the file-system structure. Most other PC products were able to read dBase files easily. I was able to read dBase files on an XP using a 1984 PC software! A true database can only be read by its own DBMS software. Management of data within the database should be done, by a dbms and dbms alone. dBase doesn't qualify for the name database!

  But no one objected and such misnomer helped the new vendors, especially the relational database vendors. Though RDBMS had dbms software, but most of them were SQL front-ends to VAX/PDP based file systems, at leats to start with.

  With this background, may I pose the same question: does JavaDB have a true DBMS inside? My own feeling is that JavaDB will be only a help db in the JavaWorld. No one expects this to be used for any serious applications where a database is needed. May be good fr simulating and testing Java applications where you want a quick and dirty database at arms length. Nothing more!

  Also there is teh question: why did Javadb go for a relational db? OO model is closer to the E_R model. And a network db based on E-R model, with some modern access commands would have been more ideal model for JavaDB. Simulating one more relational database within Java doesn't do any good!

  Posted by: manugopi on November 07, 2007 at 07:12 AM

This work is licensed under a Creative Commons License.