John Reynolds's Blog

Piercing through data abstractions

A recent request from my QA team lead brought me to this not-so-stunning conclusion. The request was pretty simple:

"Tell me which fields in our database correspond to the input fields on our HTML forms."

A reasonable request, but also a rather ironic request: By practicing object oriented design principles such as separation of concerns and data hiding, we have crafted a system where the physical database structure is irrelevant to the web-tier of our application. Our web-tier components don't "require" a relational database� the data store could be an XML store or a comma-delimited-flat-file or a psychic channeling knowledge to and from an alien civilization. This is not an inappropriate or even an unusual abstraction, just standard design practice.
But the fact remains that our data sets are in a relational database, and in fact we have legacy systems that directly interact with the same records. Testing the impact of the UI on the database tables (and visa-versa) is appropriate.

So how do I generate a mapping between specific fields on our HTML forms and specific fields in our database tables?
The primary O/R mappings are between our Entity Beans and the database. We are using Session Beans as a facade between the Entity Beans and our Struts based classes, and we are using JSP tags to render the elements on the resultant HTML pages. It is certainly possible to navigate through the code to determine the relationships between the data store and the displayed pages, but as you can see it is a tedious undertaking.

I think this is where Java metadata might come to the rescue (in the future). As I have indicated in my earlier blogs on Field Validation and on Java Process Definition, I am a big fan of metadata.

In this case, imagine that the metadata for my Entity EJB included the pertinent fact that a relational database is the data store. Expand that thought to include in the metadata the EJBQL select statement used to populate and update each field. Now add to my Fa�ade Session Bean an interface to pierces through the abstraction to gather details about the implementation� Not details that allow a consumer to take advantage of the implementation, simply details to document the implementation.
I have not quite worked through the whole process, but what I am yearning for is the ability to percolate knowledge of implementation details up to the higher layers of my application. Carry this concept further, and I should be able to create a build-time process that generates a mapping between fields on my JSPs and fields in my relational database.

This mirrors Michael Champion's comments on contextual metadata:

"We need to build tools and products that tap the small packets of structure and context and pull it into useful business information."

In the real world of today, I am going to sit down with a #2 pencil, some screen shots, our data model, and a cup of coffee. With any luck I will be able to provide the information that my QA lead needs by tomorrow morning.
Hopefully in the near future a judicious jolt of Java metadata will help me to spend my time a bit more productively.

Comments

• Law of Leaky Abstractions
  Joel Spolsky has an interesting article on "The Law of Leaky Abstractions":

  http://www.joelonsoftware.com/articles/LeakyAbstractions.html

  Posted by: clamprecht on January 02, 2004 at 01:35 PM

• Law of Leaky Abstractions
  Thanks for the link to the article. It's very apropos: "abstractions do not really simplify our lives as much as they were meant to. "

  Posted by: johnreynolds on January 02, 2004 at 02:42 PM

• Do Unit testing; not end-to-end testing.
  A recent request from my QA team lead brought me to this not-so-stunning conclusion. The request was pretty simple: "Tell me which fields in our database correspond to the input fields on our HTML forms". This is a very reasonable request. My QA team wants to initialize our relational database with a specific data set, verify that the data displays properly in the browser, and then verify that changes made from the browser show up in the relational database.

  What this example shows is that your team should not do end-to-end testing. They should test smaller units. Read up on JUnit testing frameworks to see how if you test your smaller units closely enough you don't need to test end-to-end anymore (not automatically anyway).
  Additionally; the tests fail much faster as well. Showing a much better locality of bugs.

  Posted by: zander on January 05, 2004 at 03:21 PM

• Do Unit testing; not end-to-end testing.
  Our developers do in fact use JUnit, and it is certainly a valuable part of the puzzle (easpecially when coupled with Cactus).
  Unit testing is not however a replacement for functional testing in large organizations... leaving end-to-end testing to our customers is not considered being "customer friendly" :-)

  
  

  Posted by: johnreynolds on January 05, 2004 at 07:38 PM

This work is licensed under a Creative Commons License.