The Java KeyStore API supports multiple keystore formats which include JKS( the default Java KeyStore), PKCS12, PKCS11 etc. By default when GlassFish V3 is installed the default Keystore Type is JKS and the server keystore (keystore.jks) is located in the domain config directory.  With latest GlassFish V3 builds it should be possible to define a different KeyStore Type such as PKCS11 or PKCS12 for the KeyStore type and use a corresponding PKCS11 or PKCS12 store as the server keystore.

Here are the steps to configure GlassFish V3 with a PKCS12 keystore as the server keystore. All the steps mentioned here can be achieved by using the Admin GUI however i would just show the resultant change in domain.xml

1. Install the PKCS12 (.pfx/.p12) Keystore inside the domain config dir (such as domains/domain1/config). Assuming the name of the KeyStore is s1as.p12 copy the file into the config dir

2. Update the jvm-options corresponding to keystore and truststore settings to look as follows :

           <jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>

        <jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/s1as.p12</jvm-options>

        <jvm-options>-Djavax.net.ssl.keyStoreType=PKCS12</jvm-options>

        <jvm-options>-Djavax.net.ssl.trustStoreType=JKS</jvm-options>

3.  In V3 the network-listener corresponding to secure port 8181 is disabled by default. So set the enabled attribute to true on network-listener corresponding to 8181

4.  Make sure the SSL child element under the protocol configuration for http-listener-2 looks as below (specifically remove the  cert-nickname="s1as" attribute since we now use a PKCS12 Keystore containing the Public/Private KeyPair).

             <protocol security-enabled="true" name="http-listener-2">

            <http max-connections="250" default-virtual-server="server" server-name="">

              <file-cache enabled="false" />

            </http>

            <ssl ssl3-enabled="false" />

          </protocol>

 5. If the server ceritificate contained in your PKCS12 keystore is issued by a tursted CA then make sure the CA certificate appears inside the glassfish truststore (cacerts.jks). Incase you are using self-signed certificates inside the PKCS12 keystore then make sure you import the server certificate into glassfish truststore.

6. save the changes, restart glassfish  and try to access https://localhost:8181. This should take you to the GlassFish Server Welcome page.

 For my testing i had to use the default s1as  KeyPair that comes with the GlassFish default installation (residing inside keystore.jks). So here are the steps to export the s1as keypair as a PKCS12 keystore.

  a) Export the server certificate

     keytool -export -file s1as.der -keystore keystore.jks -storepass <GF-password> -alias s1as

  b) Export the Private Key for the server in PEM (Privacy Enhanced Mail)  format. Use the KeyExport tool for doing this (download keyexport.zip here)

  c) unzip keyexport.zip to find the jar file keyexport.jar

  d) run the following command to export the private key :

    java -cp keyexport.jar com.sun.xml.wss.tools.KeyExport -keyfile s1askey.pem  -alias s1as -keystore <GF-DOMAIN-CONFIG-DIR>/keystore.jks -outform PEM -storepass <GF-password> -keypass <GF-password>

  e) Convert the DER encoded certificate from step (a) into PEM format as well

     openssl x509 -in s1as.der -inform DER -out s1as.PEM -outform PEM

  f)  Concatenate the  Certificate and PrivateKey PEM files into a single file. In mycase i would append the private key file to the certificate file. So that the file looks as follows :

-----BEGIN CERTIFICATE-----

.....

-----END CERTIFICATE-----

-----BEGIN PRIVATE KEY-----

......

-----END PRIVATE KEY-----

 g) Now use openssl again to convert the concatenated Certificate + PrivateKey into a PKCS12 keystore

openssl pkcs12 -export -in s1as.PEM -out s1as.p12

Enter Export Password: <enter GF-password>

Verifying - Enter Export Password: <enter GF-password again>

The last step above creates the s1as.p12 which is a PKCS12 keystore that can be used as the GlassFish Server Keystore.