Felipe Gaucho's Blog

October 2007 Archives

Loading Properties from XML (revisited)

My last blog about using JAXB instead of Properties for loading configuration of Java applications was a bit verbose - so, I decided to print a summary in order to facilitate the comprehension about the original proposal. I will not expose the discussion again, just present the below comparison sheet:

text=word
primitive=2
anobj.name=test
anobj.integer=3

// Read properties file.
Properties properties = new Properties();
properties.load(new FileInputStream("filename.properties"));

// Unmarshal the properties XML file into a Java Object
JAXBContext jc = JAXBContext.newInstance(
		AbstractJaxbFootprintStream.FOOTPRINT_CONTEXT, this.getClass()
				.getClassLoader());

Unmarshaller unmarshaller = jc.createUnmarshaller();
unmarshaller.setEventHandler(new FootprintConfigValidationHandler());
unmarshaller.setListener(listener);
SchemaFactory sf = SchemaFactory
		.newInstance(javax.xml.XMLConstants.W3C_XML_SCHEMA_NS_URI);
Schema schema = sf.newSchema(getClass().getClassLoader().getResource(
		"config.xsd"));
unmarshaller.setSchema(schema);

MyConfig properties = (JAXBElement) unmarshaller
		.unmarshal(new FileInputStream("filename.xml")).getValue();

// A String:
String text = properties.getProperties("text");

// Other primitive values:
int number = Integer.parseInt(properties.getProperties("primitive"));

// An object:
String name = properties.getProperties("anobject.name");
int integer = properties.getProperties("anobject.integer");
Object yourObj = new YourObj(name, integer);

// A String: 
String text = properties.getText();

// Other primitive values:
int number = properties.getPrimitive();

// An object: 
Object yourObj = properties.getMyObject();

* The title of this blog is an intentional reference to this 4 years old article.

Configuration Objects: using JAXB instead of java.util.Properties

Configurable features is a common requisite of computer systems, and the Java API provides the utility class java.util.Properties to facilitate our lives. It is very simple to use, and it is suitable for the most part of applications, but it also has some limitations - more precisely when we consider the configuration data:

1. How to check if all key/values are really present in the properties file?
2. How to check if the type of the values are consistent with the types required by the application?
3. How to represent complex types (nested properties)? For example: I have a property that is a set of properties.
4. How to represent values that are not Java primitive types?

Special remind: properties loading using the java API just reads the key/values pair in the memory , it doesn't check its consistency or any data structure related to the configuration. And more: unfortunately the Properties class is locked in the old Properties DTD and it doesn't support nested structures or complex types. It also provides a XML validation mechanism, but it is really too simple to be useful in more complex scenarios - like the one I present below.

A real world example where java.util.Properties seems not to be enough

My need for a better Properties loading mechanism started when I coded the Footprint project - Footprint is a signed PDF generator that requires three distinct set of Properties:

1. The JDBC Driver Properties: required by java.sql.DriverManager#getConnection
2. A mapping between the name of the fields in the PDF template and the name of the column names in the database.
3. The general properties used to configure the system (number of threads, resources directories, etc.).

My prime code was based on three different files, and I also thought of creating a unique properties file referring the other ones or adopting some pre-defined prefixes/sufixes in order to identify the different set of properties - like jdbc.prop1=value. After some analysis it became clear my system required a more sophisticated configuration mechanism than a collection of key/value pairs of strings. It requires nested structures and better yet if it can have validation of its configuration data. Natural conclusion: these requirements can be easily achieved with an XML file. After deciding to use XML, I evaluated some options, including different XML frameworks and also the usage of XML with the Java API. I also checked the new features of JAXB 2.0, and the fact it is already distributed with the JRE 1.6 convinced me that JAXB is a first class configuration binder. The figure below summarizes my observations about property files versus XML:

Option #1: several Properties	Option #2: XML file converted in Configuration Object through JAXB
Good: Plain text file Easy visualization of key/values pairs Simple code for reading the properties Default Java API - required classes are distributed as part of JRE (since 1.2) Bad: Several files to manage Weak validation of files structure and contents Text editors or IDEs don't help to identify problems String + primitive types representation only Error prone - it is very hard to visualize missing data Custom binder - it requires a custom code to convert String values in Java Objects Inclusion of new key/pair on demand - hard to control the mess in big teams	Good: Plain text file Only 1 file to manage Configuration data defined in a model Validation support by default using the XML Schema Configuration versioning Data types and values range configurable in the model Any Java type supported Default Java API - required classes are distributed as part of JRE (since 1.6) Full support from IDEs in editing time (highlight, validation and auto-complete) Missing data never happens Automatic binding between the configuration values in Java Objects Inclusion of new properties depends on model changing - more easy to control the mess in big teams - just keep the model locked ;) Bad: Difficulty in visualizing the information Any new configuration value impacts in a model changing

Implemented solution: binding XML properties to Configuration Object through JAXB 2.0

The full source code I used to implement my JAXB based properties mechanism is available at the Footprint SVN repository - including the ANT task required to create the XML<->Object binder using XJC. I strongly recommend you to download the footprint source code and try it a bit before checking my conclusions at the end of this blog. Inspecting the code you will start seeing how clean is the application regarding to its configuration properties. I don't have hard-coded String representing the configuration acronyms, and I don't need to verify if the values in the properties are consistent with the values required by the code, it is implicit since we are calling the methods of the Configuration Object that are homonymous to the properties names. Other comfortable feature: if I modify the XML schema or try to change the code in a way both sides become inconsistent, the IDE will notify the problem, avoiding me to crash my own code - the code will not compile before the classes and the properties model become consistent to each other, even under that stressful deadline :)

First step: you first need to create a XML Schema, and then use an ANT task to generate the XML<->Object binder classes, like the one below:

	<taskdef name="xjc" classname="com.sun.tools.xjc.XJCTask">
		<classpath refid="classpath.base" />
	</taskdef>
	
	<target name="compileschema">
		<echo message="Compiling the Curriculum XML Schema..." />
		<mkdir dir="${jaxb.classes.dir}" />
		<xjc schema="${schema}" extension="true"
			classpath="${build.dir.classes}" target="${generated.dir}"
			package="${jaxb.bindingclasses.package}">
			<produces dir="${jaxb.classes.dir}" includes="**/*.java" />
		</xjc>
		<javac srcdir="${generated.dir}" destdir="${generated.dir}"
			debug="off" source="1.5" />
		<jar destfile="lib/footprint-config.jar" basedir="${generated.dir}"
			excludes="**/*.java" />
		<delete dir="${generated.dir}" />
	</target>

Before discussing the pros and cons of the JAXB adoption, let me show some code fragments. What we need to do is to load and to use the configuration properties, what is quite easy with the newest version of JAXB. The validation of the XML against its schema is automatic, but I can also implement a finer verification of the properties data using the Event Callbacks mechanism provided by JAXB.

1. A code for reading the XML properties file:

   	@SuppressWarnings("unchecked")
   	public JAXBElement read(InputStreamReader inputStream)
   			throws Exception {
   		JAXBContext jc = JAXBContext.newInstance(
   				AbstractJaxbFootprintStream.FOOTPRINT_CONTEXT, this.getClass()
   						.getClassLoader());
   
   		Unmarshaller unmarshaller = jc.createUnmarshaller();
   		unmarshaller.setEventHandler(new FootprintConfigValidationHandler());
   		unmarshaller.setListener(listener);
   		SchemaFactory sf = SchemaFactory
   				.newInstance(javax.xml.XMLConstants.W3C_XML_SCHEMA_NS_URI);
   		Schema schema = sf.newSchema(getClass().getClassLoader().getResource(
   				AbstractJaxbFootprintStream.SCHEMA_FILE));
   		unmarshaller.setSchema(schema);
   
   		JAXBElement footprintProperties = (JAXBElement) unmarshaller
   				.unmarshal(inputStream);
   		return footprintProperties;
   		// That's it! From this point I have a Java Object I use to retrieve the configuration values
   	}
   	

2. After binding the properties to Java Object, I can manipulate its attributes as any other class:

   	private void DemoMethod(FootprintProperties properties) {
   		ConfigEmail email = config.getEmail();
   		setMsgFrom(email.getMsgFrom()); // GOODBYE hard-coded strings like prop.get("msg.from");
   	}
   

3. An optional Unmarshal Event Callbacks, to check what is being read. Observe in the code below that I can inspect every element of my configuration, which means I can stop the application loading if I detect something inconsistent. I am using this listener also to log what is being loaded. And since I have a model, I know what any fragment of my configuration file is about and I can do extra verification on the config values. Eventually I can also modify the created Object based on custom requirements.

   	@Override
   	public void afterUnmarshal(Object target, Object parent) {
   		if (target instanceof ConfigEmail) {
   			validateEmailData((ConfigEmail) target);
   		}
   		...
   	}
   		
   	// Observe I don't need to check if the msgTo attribute is present or not
   	// in my property file, the model guarantee that.
   	private void validateEmailData(ConfigEmail email) {
   		String msgFrom = email.getMsgFrom();
   		Pattern p = Pattern.compile(".+@.+\\.[a-z]+");
   		Matcher m = p.matcher(msgFrom);
   		boolean matchFound = m.matches();
   		if (!matchFound) {
   			severe(FootprintConfigUnmarshallerListener.I18N_KEY_EMAIL_MALLFORMED,
   				new String[] { msgFrom });
   		}
   	}

Conclusions

The implemented code works fine, and despite I didn't stressed too much the tests, I have some observations:

1. The loading performance is good, since a regular configuration file is never more than hundreds of values.
2. Memory impact: large sets of configuration can cause an explosion of Objects in the memory, and that is a good thing to keep in mind: each element of the XML will instantiate a different Object.
3. The complexity of the reading/writing algorithm is similar do the one required by the java.util.Properties, and after creating it once you can reuse it later.
4. The validation and verification of the configuration data are sound and robust processes - much more elegant and safe than using properties files.
5. Open the configuration example XML in your preferred IDE, and you will notice the helpful support available there, including highlight and auto-complete features.
6. Think about the natural match between the XML structure and the beans editors code out there. It is quite easy to create a GUI that presents to the user a tree with all configuration elements and its respective editors. I started a Swing prototype here, but it is still a pending task. Eventually I will use the NetBeans platform, that gives me a wonderful beans editor support, or even try JavaFX to produce the configuration editor GUI.

A polemic topic is about the large amount of Objects loaded in memory, what is controversial if you consider what happens when you load values using java.util.Properties. The properties are loaded as Strings, what in the worst case means one String object to each value. Even considering Strings are lightweight objects compared to complex types, all Strings will be used to instantiate an Object at some part of your code. One can suggest JAXB is much heavier since it loads all objects at once, while properties being used by demand will rarely cause all objects being loaded together. I agree in terms, but I think JAXB provides a better productivity - reducing the chances of mistakes and facilitating the verification of the configuration and the code. Imagine a developer using auto-complete for loading the properties, instead of checking constants somewhere or including weird hard-coded String all over the code. It is up to you to think about that.

My overall feeling about using JAXB instead of Properties is very good, I am feeling much more confident to code based on Objects than based on collection of Strings. At the end, both approaches are quite similar in terms of logic, but since I am an Object Oriented developer, I prefer to manipulate objects than lists of values - the old fashion procedural approach.

Footprint: stable release is out (codename: FOR)

After a long vacation, I finally had time to release the first stable release of Footprint :). The project is new and should evolve to something more sophisticated in the next few months. For now, we are proud to provide the community a simple tool to enhance the quality of JUG events or perhaps bigger or more commercial events. I hope you follow our Footprint. * The project's license is LGPL and the full source code is available at the SVN repository.

The Footprint project

As explained in my last blog about the Footprint, the aim of the project is to provide an easy way of delivering certificates to JUG meetings attendants. The basic user story is the following:

1. The JUG promotes an event.
2. During the event, a JUG representative collects the name and email of the participants.
3. After the event, all participants receive a certificate of presence - signed PDF file - through email.
4. The generated certificates can be validated through the public key of the JUG, published in the JUG web site.

How it works?

The algorithm of footprint:

1. Reads an XML config file, and uses JAXB to construct an Object representation of the configuration. The configuration file contains several definitions, such as JDBC connection, PDF template and keystore details.
2. Reads the PDF template, previously created with OpenOffice or other PDF generator tool.
3. Access the JDBC connection to read the JUG members data. For each addressee:
   1. Fill the PDF template with the member data (name, address, etc.)
   2. Generate a new PDF file with the member information.
   3. Sign the new PDF
   4. Send the new file by email to the addressee

The security part of the project is based on the Bouncy Castle's library, and all PDF manipulation is done with the fantastic iText library. Other basic features are the logging over all operations and the internationalization of all classes.

How to run the example?

The release contains a README.txt file with brief instruction on how to start using Footprint, but in short you must:

1. download the release and extract it in your computer.
2. Edit the file resources/configExample.xml and change the user and password of the SMTP server:

   <email smtp.user="EMAIL_ACCOUNT_LOGIN" smtp.password="EMAIL_ACCOUNT_PASSWORD" msg.from="EMAIL_ACCOUNT_LOGIN" smtp.transfer.protocol="smtp"
   	smtp.star.ttls.enable="true" smtp.host="SMTP_HOST" smtp.content.type="text/plain"
   	smtp.auth="true" socks.proxy.port="" socks.proxy.host=""
   	msg.subject="Footprint Certificate - DEMO"
   	msg.body="You can write a message body here.. or customize it through the code, check the demo file..." />
    />
   

   The example comes pre-configured to access the gmail server, but you can change it to any SMTP server. You can also configure the proxy information in case you are behind a firewall. You can also try a first time without configuring the SMTP settings, it will generate the signed PDF without sending it through email.
3. Edit the file demo.csv in order to include an email address you can verify later :). You can include several new lines in this file in order to observe the system generating multiple PDFs.
4. Open a console and run the following command:

   java -cp .;lib\bcprov-jdk16-136.jar;lib\csvjdbc.jar;lib\footprint-config.jar;lib\footprint-core.jar;lib\itext-2.0.2.jar;lib\mail.jar FootprintDemo

That's it - if it fails for any reason, please don't hesitate to send me an email with the failure description.

How to change the configuration file?

Documentation is an issue by design of our project. Since we don't have many collaborators, we always prioritize the implementation (code first). But you can check the configuration schema, and if you have doubts about its structure, please leave a comment below this page.

This work is licensed under a Creative Commons License.