Data Abstractions are wonderful things, except when they aren't.

A recent request from my QA team lead brought me to this not-so-stunning conclusion. The request was pretty simple:

"Tell me which fields in our database correspond to the input fields on our HTML forms."

This is a very reasonable request. My QA team wants to initialize our relational database with a specific data set, verify that the data displays properly in the browser, and then verify that changes made from the browser show up in the relational database.

A reasonable request, but also a rather ironic request: By practicing object oriented design principles such as separation of concerns and data hiding, we have crafted a system where the physical database structure is irrelevant to the web-tier of our application. Our web-tier components don't "require" a relational database� the data store could be an XML store or a comma-delimited-flat-file or a psychic channeling knowledge to and from an alien civilization. This is not an inappropriate or even an unusual abstraction, just standard design practice.
But the fact remains that our data sets are in a relational database, and in fact we have legacy systems that directly interact with the same records. Testing the impact of the UI on the database tables (and visa-versa) is appropriate.

So how do I generate a mapping between specific fields on our HTML forms and specific fields in our database tables?
The primary O/R mappings are between our Entity Beans and the database. We are using Session Beans as a facade between the Entity Beans and our Struts based classes, and we are using JSP tags to render the elements on the resultant HTML pages. It is certainly possible to navigate through the code to determine the relationships between the data store and the displayed pages, but as you can see it is a tedious undertaking.

I think this is where Java metadata might come to the rescue (in the future). As I have indicated in my earlier blogs on Field Validation and on Java Process Definition, I am a big fan of metadata.

In this case, imagine that the metadata for my Entity EJB included the pertinent fact that a relational database is the data store. Expand that thought to include in the metadata the EJBQL select statement used to populate and update each field. Now add to my Fa�ade Session Bean an interface to pierces through the abstraction to gather details about the implementation� Not details that allow a consumer to take advantage of the implementation, simply details to document the implementation.
I have not quite worked through the whole process, but what I am yearning for is the ability to percolate knowledge of implementation details up to the higher layers of my application. Carry this concept further, and I should be able to create a build-time process that generates a mapping between fields on my JSPs and fields in my relational database.

This mirrors Michael Champion's comments on contextual metadata:

"We need to build tools and products that tap the small packets of structure and context and pull it into useful business information."

The answers to many questions are there in the code, we just need better mechanisms for extracting them.

In the real world of today, I am going to sit down with a #2 pencil, some screen shots, our data model, and a cup of coffee. With any luck I will be able to provide the information that my QA lead needs by tomorrow morning.
Hopefully in the near future a judicious jolt of Java metadata will help me to spend my time a bit more productively.

---

(Cross posted at The Thoughtful Programmer)