Automated functional tests: taking your QA database back to a well known state
Automated functional tests are key to ensure the quality of large applications in incremental development processes. In contrast with unit tests where each test is supposed to be independent from the outside environment, functional tests are really integration tests: in order for them to run properly the process must start from a well known state.
The majority of enterprise applications use a relational database in order to store application state. As one starts to automate the functional tests a problem quickly arises: your automated tests will modify the QA database and that may have an impact in the future automated test runs. For instance, you write a test for deleting a particular record and it runs fine, but that means in the next run the record won´t be there to be deleted. Of course you can write your automated tests in order to make sure this is not a problem (for example, by ordering your tests so that the record deleted in the delete test is first inserted by a previous test). But that makes the automated tests a lot more difficult to write and run reliably.
The ideal solution for this problem (in my oppinion) would be to reset the status of the QA database to a well known state just before starting the automated tests. That´s exactly what I did in a recent customer project using Ant, a little scripting and PostgreSQL:
1 - Create a template QA database
This is the database state you want to reset to at the beginning of the automated functional test suite. In PostgreSQL it can be done using a few commands:
createdb -U postgres -O <owner> -T template0 AutoQADB
pg_dump -U postgres -f source_dump.sql -b -F c QADB
pg_restore -U postgres -d AutoQADB source_dump.sql
In this example QADB is your current QA database, AutoQADB is a copy of this database to be used exclusively by the automated test process.
2 - Create a database reset script
This could be done entirely using Apache Ant, but since in this project the development environment runs under Microsoft Windows I found it easier to create a simple BAT file (Ant has some problems with the exec task on Windows). The purpose of this script is to recreate the AutoQADB database when required:
cmd.exe /c "C:\glassfish-3.0.1\bin\asadmin.bat" stop-domain domain1
"c:\PostgreSQL\8.4\bin\pg_dump" -U postgres -f source_dump.sql -b -F c QADB
"c:\PostgreSQL\8.4\bin\dropdb" -U postgres AutoQADB
"c:\PostgreSQL\8.4\bin\createdb" -U postgres -O <owner>-T template0 AutoQADB
"c:\PostgreSQL\8.4\bin\pg_restore" -U postgres -d AutoQADB source_dump.sql
"C:\glassfish-3.0.1\bin\asadmin.bat" start-domain domain1
In order for this script to work it is important that nobody is connected to the AutoQADB as the script drops it and re-creates it again as a copy of QADB. Since in this project uses GlassfishV3, I included commands to stop the Glassfish QA instance (first line) and start it again once the database is ready (last line).
3- Invoke the reset script in your ant build before running the automated tests:
<target name="restore-QA-database" >
<exec executable="cmd" >
<arg value="/c" />
<arg value="refresh_QA.bat" />
In this example the reset script is called refresh_QA.bat. The run-qui-tests target runs all the tests in the automated functional test suite. By declaring that it depends on the restore-QA-database target we make sure the tests will always start with the desired database state.
The specific examples shown here work with PostgreSQL and GlassfishV3, but the technique can be adapted to work for other combinations of database and application servers.