John Reynolds's Blog: Patterns Archives

The conceptual goal of this article is to convince component developers that the rules that govern form validation should be embedded in the form definition rather then spread between business and presentation logic. This approach (which I have advocated before) simplifies the mapping between requirements and implementation, and leads to code that is easier to maintain.

Forms are first class citizens in the world of Business.

Almost all Business Processes begin with the submittal of a Form, and seldom can a Business Process proceed until the Form's input has been validated against a set of rules.

Unfortunately, in the world of User Interface Components the Form has lost its first class status. The term Form has been usurped to refer to a set of Data Fields that are presented to a user on a single screen rather then to an underlying Business Object.

Consider a web-based application for preparing US Income Taxes: Several screens may be necessary to complete a 1040 Income Tax Form, but from the developer's perspective each screen will submit a unique HTML Form.

To avoid confusion, for the remainder of this article I will use the term "Business Form".

Most UI Component Frameworks are Data Field oriented. Controls such as text boxes and drop-down lists are added to screens to gather information from a user. To insure that the user supplies correct information, the controls either constrain the user's input (as with a drop-down list), or the input data is validated against a set of rules for the individual field, or a combination of both techniques is used. Each field has its own validation rules, without respect to the field's context within an underlying Business Form.
This approach is by no means limited to web-based applications, Microsoft's Visual Basic has been renowned for providing field validations of this nature for years.

In the web world, validation of data fields is often a two stage affair. It is always best to provide feedback to a user's input as quickly as possible, so often input data is validated on the client side (this also has the advantage of reducing round-trips to the server to validate data). Data validation must always be performed on the server side for a number of reasons. Client-side validation cannot be trusted (JavaScript may be turned off, or the HTML form may be "spoofed"), and some validations require knowledge that isn't available on the client (such as verifying that a Zip Code matches a City/State combination).

Java Server Faces and Form Validation

Java Server Faces is a relatively new UI Component Framework for developing web-based Java applications. Presentation and Validation logic is encapsulated into reusable Controls to greatly reduce the complexity of building HTML based user interfaces. From its inception, JSF was designed for use with graphical design tools, and already some very promising "Visual Basic-like" development environments have been released.

Unfortunately, JSF has perpetuated the Data Field orientation of earlier UI Component Frameworks.

In JSF, validation rules are defined on a field-by-field basis. To illustrate this point, below is a simple snippet from a JSP page that incorporates a JSF "input_text" field.


<f:view>
<h:form id="myForm">
<table>
  <tr>
    <td>Social Security Number:</td>
    <td>
      <h:input_text id="borrowerSSN" 
        value="#{loanAppBean.borrowerSSN}"
        required="true"
      />
    </td>
    <td>
      <!-- error message if borrowerSSN not supplied -->
      <h:messages for="borrowerSSN"/> 
    </td>
  </tr>
<!-- Additional fields deleted for clarity -->
</table>
</h:form>
</f:view>

In the above snippet, the markup indicates that the user is required to supply a "borrowerSSN" (the attribute "required" is set to true). If the user submits the HTML form without supplying a "borrowerSSN", an error message will be displayed next to the input field.

Admittedly, this is a straightforward and almost trivial example of field validation, but it serves to illustrate the inherent problems with Data Field oriented validation. In the above example, "borrowerSSN" represents a single data field of a Business Form. Possibly, all of the fields of this Business Form reside on a single web page, but possibly the Business Form requires several pages to complete. Regardless, the Business Rule:"borrowerSSN cannot be null" is embedded on this web page. If the Business Rules change, this web page will have to change. There is not a clean mapping from the Business Requirement to the implementation of the requirement.

For trivial applications, Data Field oriented validation has few drawbacks, but consider the potential problems for applications that rely on external services.
Imagine an application that gathers the information necessary to submit a Loan Application to a Lender. The Lender provides a Web Service that accepts XML Loan Applications, and your task is to create a web-based front-end for use by borrowers. Fortunately, the Lender supplies a thorough XML Schema that defines the acceptable values, so the task of creating a UI with proper validation is relatively straightforward.

Problems begin when the Lender's Business Rules change; Returning to the previous JSF snippet, assume that "borrowerSSN" is part of the Loan Application, and that the field is "required". Assume that Federal regulations change and it becomes illegal to require a borrower to supply an SSN. The Lender's web service and your web application are now in violation of Federal law, so you'll both have to hustle to change your code. The Lender must update the XML Schema of their web service, and you must now go through all the pages of your application to make the proper changes.

This simple example illustrates why Business Form oriented validation is preferable to Data Field oriented validation. Concentrating the rules for validating a Business Form in a single interface requires a bit more "plumbing", but results in a solution that is easier to maintain then the Data Field oriented approach.
Note that consolidating the Rules for validation does not require consolidating the processing of those rules. All processes that operate on a Business Form should have access to the Form's declarative rules. If a Rule is simple enough to be evaluated by Javascript, then there is no reason why a JSF component shouldn't generate client-side Javascript when the page is rendered.

Returning once again to our JSF snippet, consider the following changes:

<f:view>
<h:form id="myForm">
<table>
<tr>
<td>Social Security Number:</td>
<td>
<h:input_text id="borrowerSSN" 
      value="#{loanAppBean.borrowerSSN}"

      <!-- get validation rules from the Form -->
      validationRules="#{loanAppBean.borrowerSSNValidationRules}" 
      <!-- replaces: required="true" -->
/>
</td>
<td>
<!-- error message if borrowerSSN not supplied -->
<h:messages for="borrowerSSN"/>
</td>
</tr>
<!-- Additional fields deleted for clarity -->
</table>
</h:form>
</f:view>

This mock-up is intentionally verbose to make a point: JSF works by using Java introspection to get values from Java Beans. The following line in effect invokes the "getBorrowerSSN()" method on the Java Bean that corresponds to "loanAppBean":

 
      value="#{loanAppBean.borrowerSSN}"

A similar approach could be used to obtain validation rules from a Java Bean. Conceptually, the following line gets the "borrowerSSN" validation rules from the "loanAppBean&" and passes them to the JSF "input_text" component.

      validationRules="#{loanAppBean.borrowerSSNValidationRules}"

As I mentioned earlier, this example is more verbose then need be. If a convention is adopted to define Rules in BusinessFormBeans, Java introspection can be used to obtain Values and Rules given a single field identifier.

Validation of Inter-Dependent Fields

Thus far, I've only covered validation of single fields. Business Forms frequently have Rules that govern dependencies between multiple fields. For example, if a Business Form includes a Start Date and an End Date, there will certainly be a Rule that governs both fields (the Start must be earlier then the End). Validation of this nature is often referred to as "Form Based Validation".

JSF does not explicitly handle Form Based Validation. The JSF Validation Model is strictly Component oriented. A single Component can gather multiple data values, but dependencies between Components must be handled by implementing custom Event Handlers.

JSF's hooks for implementing custom Form Based Validation are the ValueChangedListener and the ActionListener classes. A custom ValueChangedListener can be registered on any Component. Whenever the value of the Component is changed, the Listener will be invoked.
A custom ActionListener can be registered on any UICommand Component (like a "submit" button). When the Component is activated, the Listener is invoked.

With these two building blocks, it's possible to perform any necessary Form Based Validation, but making this work seamlessly with the standard JSF Validation Model requires finesse. Consider the following snippet:

<f:view>
<h:form id="myForm">
<table>
  <tr>
    <td>Start Date:</td>
    <td>
      <h:input_text id="startDate" 
         value="#{loanAppBean.startDate}"
         required="true"
      />
    </td>
    <td>
      <!-- error message if startDate not supplied or > endDate -->
      <h:messages for="startDate"/> 
    </td>
  </tr>
  <tr>
    <td>end Date:</td>
    <td>
      <h:input_text id="endDate" 
        value="#{loanAppBean.endDate}"
        required="true"
      />
    </td>
    <td>
      <!-- error message if endDate not supplied or < startDate -->
      <h:messages for="endDate"/> 
    </td>
  </tr>
</table>
</h:form>
</f:view>

The standard JSF Validation will populate the error messages if either Date is null, but a custom Event Handler must be implemented to populate the error messages if the endDate is before the startDate. The rules that apply to the fields are declared in multiple locations within the code base.

The process would be much simpler if a BusinessFormBean (a Java Bean that includes "getter" functions for declarative rules that apply to the form and each of its fields) were registered on the JSF form itself:

<f:view>
<h:form id="myForm" businessFormBean="loanAppBean" >
<table>
  <tr>
    <td>Start Date:</td>
  <td>
      <h:input_text id="startDate" 
        value="#{loanAppBean.startDate}"
        required="true"
      />
    </td>
    <td>
        <!-- error message if startDate not supplied or > endDate -->
        <h:messages for="startDate"/> 
    </td>
  </tr>
  <tr>
    <td>end Date:</td>
    <td>
      <h:input_text id="endDate" 
        value="#{loanAppBean.endDate}"
        required="true"
      />
    </td>
    <td>
        <!-- error message if endDate not supplied or < startDate -->
        <h:messages for="endDate"/> 
    </td>
  </tr>
</table>
</h:form>
</f:view>

Of course, for this technique to work "BusinessFormBean" would have to implement a standard interface for exposing Form Based Validation Rules to the JSF form component.

JSF is not unique

I hasten to mention that JSF is not alone in its approach to field validation. The Tapestry Web Component Framework (which I happen to think is great) provides Data Field and Form Based validation support that is philosophically similar to JSF. As with JSF, components can be developed for Tapestry to support Business Form oriented validation as I have suggested in this article. I fervently hope that all Component Frameworks consider adopting the approach that I am advocating in this article.

There are a number of trends that seem to be leading us towards the need to standardize a "Java Bean-like" interface for retrieving Business Form Rules:

XML technologies such as XMLSchema and Relax NG provide declarative validation rules for XML Documents. Rules for Web Services and UI based apps should be shared.
Open Source Rules Engines such as Drools are rekindling interest in Declarative Programming.
The W3C XForms standard defines forms in sufficient detail that browsers can generate self-validating user interface controls without explicit scripting. The definition of these rules should be independent from the decision to use XForms.
Tools for generating User Interfaces should support using common Business Rules across multiple applications. The definition of Business Forms within IDEs should be seperate from the generation of user interfaces and page flows (ala Apache Beehive).
Sun has just started the Java Server Faces Implementation Project and the JDesktop Network Components Project. This is a great opportunity for synergy and consolidation between the two projects in the area of form validation.

Simplify the Mapping from Requiements to Implementation

Component Frameworks have a lot to offer, but they can be better if they stick to presentation and rely on Business Forms to provide validation rules. I hope that I have demonstrated that JSF could be enhanced to consolidate the Rules that govern Business Forms, and that Rule consolidation simplifies the mapping between requirements and implementation. Toolsets that cleanly seperate the definition of Business Forms (and Business Processes) from the creation of User Interfaces and Page Flows will result in code that is easier to maintain and to reuse.

At the very least, I hope I've got a few more folks thinking along these lines...

Update: There are several threads on Sun's JSF forum relating to this issue:
Recommended practice for handling page level validation?
Writing a validator that validates multiple components
Validating several fields within the same validator class
Philosophical Question: validation of input data

Anti-Pattern - The Swiss Army Effect

Posted by johnreynolds on May 06, 2004 at 11:37 AM | Permalink | Comments (6)

When I was old enough to join the Cub Scouts, my father got me a really impressive Swiss Army Knife. For an 8 year old boy this gift was pretty cool, and I proudly lugged the thing around with me on all of our camping trips. I cannot recall all the tools, but there were several blades, screw drivers, can openers, and I distinctly remember a corkscrew.

After the "new" had worn off, the knife disappeared somewhere into the bowells of a dresser drawer. I didn't lose the knife, I just discovered that it wasn't all that practical. I never did use all the blades, and the tools that I did use were encumbered by all the superfluous ones.

I'm sure that you figured out where I was heading with all this when you read the title of this blog. Don't get trapped by the Swiss Army Knife pattern. Don't add too many "blades" to your applications. Build focused tools that solve specific problems.

All of that is true, but it's also far from controversial. Why waste time blogging about it?

Why indeed? Because I want to make a different point. In many ways, Java is becoming a great big Swiss Army Knife.

I am not real happy with the oposition by IBM, BEA, and Oracle to the JDO 2.0 jsr, but embedded in their arguments is a valid point ( see Three votes no on JDO by Daniel H Steinberg ). The J2EE specification is huge and Java development is complicated by too much flexibility. In this specific case, EJB and JDO both specify a persistence mechanism. EJBQL and JDOQL greatly overlap and both are subsets of JDBC's capabilities. Why not agree on a common core and deprecate the duplications?

I tried to make this point in my earlier blog: Make JDO the "P" in CMP. JDO and CMP should be complementary rather then competing solutions

The EJB/JDO overlap is just one symptom of Java's march to complexity. Perhaps we need to expand the mandate of the JCP to include consolidation and deprecation rather than just expansion. Our knife already has plenty of blades ;-)

Piercing through data abstractions

Posted by johnreynolds on January 02, 2004 at 06:29 AM | Permalink | Comments (4)

Data Abstractions are wonderful things, except when they aren't.

A recent request from my QA team lead brought me to this not-so-stunning conclusion. The request was pretty simple:

"Tell me which fields in our database correspond to the input fields on our HTML forms."

This is a very reasonable request. My QA team wants to initialize our relational database with a specific data set, verify that the data displays properly in the browser, and then verify that changes made from the browser show up in the relational database.

A reasonable request, but also a rather ironic request: By practicing object oriented design principles such as separation of concerns and data hiding, we have crafted a system where the physical database structure is irrelevant to the web-tier of our application. Our web-tier components don't "require" a relational database� the data store could be an XML store or a comma-delimited-flat-file or a psychic channeling knowledge to and from an alien civilization. This is not an inappropriate or even an unusual abstraction, just standard design practice.
But the fact remains that our data sets are in a relational database, and in fact we have legacy systems that directly interact with the same records. Testing the impact of the UI on the database tables (and visa-versa) is appropriate.

So how do I generate a mapping between specific fields on our HTML forms and specific fields in our database tables?
The primary O/R mappings are between our Entity Beans and the database. We are using Session Beans as a facade between the Entity Beans and our Struts based classes, and we are using JSP tags to render the elements on the resultant HTML pages. It is certainly possible to navigate through the code to determine the relationships between the data store and the displayed pages, but as you can see it is a tedious undertaking.

I think this is where Java metadata might come to the rescue (in the future). As I have indicated in my earlier blogs on Field Validation and on Java Process Definition, I am a big fan of metadata.

In this case, imagine that the metadata for my Entity EJB included the pertinent fact that a relational database is the data store. Expand that thought to include in the metadata the EJBQL select statement used to populate and update each field. Now add to my Fa�ade Session Bean an interface to pierces through the abstraction to gather details about the implementation� Not details that allow a consumer to take advantage of the implementation, simply details to document the implementation.
I have not quite worked through the whole process, but what I am yearning for is the ability to percolate knowledge of implementation details up to the higher layers of my application. Carry this concept further, and I should be able to create a build-time process that generates a mapping between fields on my JSPs and fields in my relational database.

This mirrors Michael Champion's comments on contextual metadata:

"We need to build tools and products that tap the small packets of structure and context and pull it into useful business information."

The answers to many questions are there in the code, we just need better mechanisms for extracting them.

In the real world of today, I am going to sit down with a #2 pencil, some screen shots, our data model, and a cup of coffee. With any luck I will be able to provide the information that my QA lead needs by tomorrow morning.
Hopefully in the near future a judicious jolt of Java metadata will help me to spend my time a bit more productively.

Field Validation Thoughts

Posted by johnreynolds on November 03, 2003 at 07:04 AM | Permalink | Comments (8)

Here at my company our business is all about forms. I’m not talking about HTML forms; I’m talking about business forms such as loan applications, tax reporting statements, etc.

We’ve got quite a collection of applications that deal with our forms, and we’ve implemented them in everything from COBOL to Java. We’ve got desktop based applications, browser based applications, mainframe batch processes, stored procedures and web services. The one business requirement that is common among these applications is this; the rules that govern what constitutes valid input for a specific form must always be the same.

This is by no means a unique situation. For example, the rules governing an IRS Form 10-40ez don’t change just because you’re using the web-based Turbo Tax rather then the desktop edition (If they did, Intuit would be in a world of trouble).

Given that all of the forms that my company deals with are well defined (and heavily regulated), one can easily envision a common repository of knowledge from which all of our applications programmatically draw the information necessary to validate field values. If the definition of one of our forms is amended (perhaps due to a regulatory change) the change will be picked up by all of our applications to insure consistent form processing across our enterprise.

Sadly, we have no such facility. We have a substantial knowledge base regarding our business, both in terms of human resources and extensive documentation, but we do not have a machine parse-able equivalent. When the definition of a form changes, we have to go through all of our systems and manually update them, and a considerable effort is expended to make sure that everything stays consistent.

This issue is one of my pet peeves, and I’m not alone. A few months back there was a good discussion on this subject on the Java Server Faces forum, and much of this blog owes thanks to the folks who chimed in at that time.

I am concerned that we Java folks are missing the boat in not generally recognizing the linkage between field validation and the underlying business forms. To illustrate my point, consider how field validation will be handled in Java Server Faces.

JSF encourages separation of concerns. Architects are encouraged to compartmentalize the design into Model, View (Presentation), and Controller concerns, and authors can be assigned to each layer. This is great.

What’s not so great (in my view) is that the current design of the JSF validation tags assigns the responsibility for defining field validations to the page author. To validate a field value, the pages author “registers the validator on a component by nesting the validator's tag within the component's tag" (excerpt from the JSF tutorial).

This bothers me because it requires the page author to have knowledge about a field’s meaning in order to select the correct validation tag.

I am not concerned that field validation is performed in the presentation layer. Validation should generally occur as close to the user as possible to provide the best possible feedback when an error is detected. I am concerned that validation is defined in the presentation layer.

Elsewhere in the JSF tutorial the responsibility for designing forms is assigned to the model author (as it should be). The form designer defines all of the fields that make up the form bean, and most likely has the knowledge of what values are acceptable for each field.

Consider the process of developing and maintaining an application: How will the rules that govern a form’s fields be conveyed from the form designer (model author) to the page designer (presentation author)? Unless this process is mechanical, it will be prone to error.

Perhaps we need a convention for specifying validation rules within form bean definitions... We now have the Java Bean getXXX and setXXX for dealing with the value of property XXX; perhaps we need "getContraintsXXX".

With such a convention, we could develop JSF validation tags to extract rules from the model, rather then having to communicate the rules to the page authors and running the risk of introducing errors. In addition, if the rules that govern a form’s fields reside with the form definition it will be easier to insure that all of the views remain consistent (even views that are web services rather then HTML).

Note that this solution only partially addresses my base concern since it deals with the HTML forms passed to the user rather then the underlying business forms, but it's a start.

Philosophically, the Struts Validator is closer to my preferred approach. With the Struts Validator, form validations for a specific application are specified in a single file named validator.xml. Unfortunately (from my perspective) the rules are defined separately from the target form, and from a packaging perspective I have not figured out how to bind the rules to every application that uses a specific form.

Another approach that I find interesting is the XForms model for constraining values. With XForms, rich declarative validation information can be included in each form definition. For a company like ours, I can envision creating detailed XForms definitions for all of our business forms, and letting those drive validation throughout the process.

Here is a summary of my thoughts:

Field validation of business forms is an important concern of many applications
The business owners should specify the field validation rules - this is business logic, not presentation logic
The validation rules associated with fields should be specified as part of the model – the rules are tied to the meaning of the data, and the meaning should remain the same regardless of the presentation
There may be presentation specific rules that are associated with a specific implementation, but the model’s rules should not be violated - the model's rules should percolate up to the presentation layers
We need a standard for specifying field constraints within form definitions, and for accessing those definitions - XML web services standards and XForms might provide good starting points

Update: See my later blog on JSF and Field Validation

Powered by
Movable Type 3.01D

java.net RSS Feeds