I experienced a problem where a client application accessing a server was unable to use the server to subsequently access a MySQL database.

However, unit testing on the server side allowed the MySQL data access.

What was happening to the server when using the remote connection then and why were SQL connection failure errors like the following being thrown?

java.sql.SQLException: Server connection failure during transaction. Attempted reconnect 3 times. Giving up.

and ...

Caused by: java.sql.SQLException: Server connection failure during transaction.
Attempted reconnect 3 times. Giving up.
at com.mysql.jdbc.Connection.createNewIO(Connection.java:1704)
at com.mysql.jdbc.Connection.(Connection.java:491)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:346)

The interesting clue was that the database access worked fine when the client accessed a web service version of the server as opposed to an RMI-based server.

Okay then so it is an RMI problem... However, no permission denied exceptions are being thrown (i.e. i thought my policy file was correct).

Creating another unit test determined that if I didn't set the RMISecurityManager everything was cool. Must be a permission problem then. But the expected result would be an exception of the form..

Caused by: java.security.AccessControlException: access denied (java.io.FilePermission data/logging_info_level.conf read)

Not so!

The life-saver was in William Grosso's book, "Java RMI", that mentioned that you can watch the security manager at work by setting the java.security.debug system property. By adding it to my server code (monitoring for "access failure"), I was able to track this output

access: access allowed (java.util.PropertyPermission file.encoding read)
access: access allowed (java.util.PropertyPermission file.encoding read)
access: access allowed (java.util.PropertyPermission file.encoding read)
access: access denied (java.util.PropertyPermission file.encoding read)
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Thread.java:1071)
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:259)
at java.security.AccessController.checkPermission(AccessController.java:401)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:542)
at java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1291)
at java.lang.System.getProperty(System.java:572)
at com.mysql.jdbc.MysqlIO.checkForCharsetMismatch(MysqlIO.java:1971)
at com.mysql.jdbc.Connection.initializePropsFromServer(Connection.java:2410)
at com.mysql.jdbc.Connection.createNewIO(Connection.java:1662)
at com.mysql.jdbc.Connection.(Connection.java:491)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:346)


It still wasn't crystal clear though as why would the access work for "file.encoding" above and then fail below. Fortunately, "google-caching" found a webpage that once had the source code for the com.mysql.jdbc.MysqlIO class.

The error was in fact here

if (encodingToCheck == null) { encodingToCheck = System.getProperty("file.encoding"); }

It was the get property on "file.encoding" from this jar and adding this line to my security.policy file fixed the connection problem

permission java.util.PropertyPermission "file.encoding", "read";

So... the practical lesson learned was, if you are having problems with RMI, set the security debug property and look to see if everything can access what it needs to

(Probably not the most revolutionary result, but gtna "googling-to-no-avail" on these exceptions / this problem made me want to put this story on the web in some form)