Skip to main content

Summary of Proprietary Features in SAAJ RI 1.3.4

Posted by kumarjayanti on December 10, 2009 at 12:09 AM PST

In this post i would like to provide a brief summary of some of the Propietary Features and implementation details of SAAJ 1.3.4 that are not necessarily related to the SAAJ API specifications. 

Creating SOAPMessages with Very Large XML Payload

The JavaDoc  of the API method SOAPMessage.saveChanges() poses a problem for the implementation by specifying that "All MIME headers in a message that is created for sending purposes are guaranteed to have valid values only after saveChanges  has been called".  Specifically the RI would try to set the Content-Length header after a  saveChanges() call and for that it would try to buffer the message contents. This prevents the RI from creating a message with a huge payload. There would be an exception of the following form  when the test method below is executed :

Caused by: java.lang.OutOfMemoryError: Java heap space

        at com.sun.xml.messaging.saaj.util.ByteOutputStream.ensureCapacity(ByteOutputStream.java:111)

        at com.sun.xml.messaging.saaj.util.ByteOutputStream.write(ByteOutputStream.java:95)

        at com.sun.xml.messaging.saaj.util.JAXMStreamSource.(JAXMStreamSource.java:63)

        at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.setContent(SOAPPartImpl.java:243)

        at com.sun.xml.messaging.saaj.soap.MessageImpl.init(MessageImpl.java:406)

So SAAJ 1.3.4  has introduced a System Property : saaj.lazy.contentlength which would allow the RI to set the Content-Length header lazily when the SOAPMessage is finally being serialized. While setting this property lazily violates the SAAJ API javadoc statement above, it would allow handling of large payloads by the RI.

 public static void testSAAJIssue50() throws Exception {

        System.setProperty("saaj.lazy.contentlength", "true");

        MessageFactory mf = MessageFactory.newInstance();

        MimeHeaders hdrs = new MimeHeaders();

        hdrs.addHeader("Content-Type", "text/xml");

        //bigmessage.xml is a BIG SOAP Envelope

        SOAPMessage created = mf.createMessage(hdrs, new FileInputStream(new File("bigmessage.xml")));

        created.saveChanges();

        String[] cls = created.getMimeHeaders().getHeader("Content-Length");

        if (cls != null) {

           System.out.println("After saveChanges() Content-Length =" + cls[0]);

        } else {

           System.out.println("After saveChanges() Content-Length =" + 0);

        }

        created.writeTo(new FileOutputStream(new File("bigmessage1.xml")));

        cls = created.getMimeHeaders().getHeader("Content-Length");

        System.out.println("After writeTo() Content-Length =" + cls[0]);

    }

Executing the above test after setting the System property will produce an output as follows :

After saveChanges() Content-Length =0

After writeTo() Content-Length =77376973

Note: The SAAJ RI design did not have any Configuration support and due to the lack of time i resorted to the use of  system properties for enabling this proprietary behavior. In future i may think of adding some Configuration support though it is again unlikely to be something in the standard SAAJ-API specification.

Enhancements in the RI for Handling Very Large MIME Messages

SAAJ RI by default cannot handle incoming SOAP Messages having Large Attachments. See issue 31 for more details. The MimeMultipart representation in SAAJ RI borrowed from the original JavaMail implementation cannot handle very large MimeParts since it stores them in Memory and the SAAJ API AttachmentPart directly refers to the MimePart. The SAAJ API is also streaming un-friendly since it has a methods  SOAPMessage.getAttachments()  and SOAPMessage.getAttachments(...).  So what was required was a MimeMultipart implementation that can handle large Mime Parts. The MimePull project uses several smart techniques including the use of the File-System and split-representation (part memory and part file-system) to handle large Mime-Parts.  In SAAJ RI 1.3.4 one can instruct the SAAJ RI runtime to use the MimePull parser by setting the system property   "saaj.use.mimepull" to "true".  This would allowing handling of  large attachments in incoming messages. The comments in  issue 31 shows a sample test code which makes use of this property. 

One thought was to make the use of MimePull the default in SAAJ RI,  however i heard from the MimePull project lead that one thing it does not handle right now is any kind of non-default Content-Transfer-Encoding (such as base64 for example).  Furthermore the MimePull API is meant purely for parsing incoming Mime Messages (and not for creating a Mime Package with Large Attachments) whereas one can do crazy things with the SAAJ API like setting the content of a received attachment part again and recreating a new SOAPMessage etc. While the latter issue has been handled the  issue of Content-Transfer-Encoding prevented me from making the use of  MimePull as the default. In future we should see the use of  MimePull as the default in SAAJ RI. 

The MimePull parser has configuration support that allows one to configure the Limits and Thresholds but the SAAJ RI just creates and uses a default configuration object. In future we could consider exposing the MimePull parser configuration in SAAJ RI.

When using the MimePull feature make sure that you close the InputStream of the Large AttachmentPart after you are done with it, this should allow the MimePull runtime to release any resources that the MimePart associated with this attachment is holding onto. For example :

AttachmentPart ap = ..... //get the attachment part from the SOAPMessage

InputStream is = ap.getRawContent();

OR

InputStream is = ap.getDataHandler().getInputStream();

try {

//use the inputstream

} finally {

  //finally close the stream

    try {

      is.close();

   }catch (IOException e) {

  }

}

When using getRawContentBytes() api, the SAAJ RI would close the inputStream. There was a bug that ap.clearContent() does not call close() on the MimePart. This has been fixed in the latest trunk after the 1.3.4 release and you can obtain the latest SAAJ 1.3.4 update bits  (saaj.1.3.4.zip) with this fix from here.

 As early as SAAJ 1.3.2,  the SAAJ RI introduced the use of the Boyer-Moore algorithm for efficient Mime-Boundary Parsing of a Mime Packaged Message. Prior to this SAAJ was using the Mime Parsing code present in JavaMail. The switch to Boyer-Moore provided 20-30%  improvement in Mime-Parsing performance. The Boyer-Moore algorithm was then re-implemented in JavaMail and lately the MimePull project uses the same Boyer-Moore implementation as in JavaMail.

However when i introduced the Boyer-Moore implementation in SAAJ 1.3.2 as the default, i was afraid of having missed some boundary conditions (not detected by our existing tests), especially i found that the original JavaMail implementation was somewhat lenient in that it accepted messages which had minor violations w.r.t  (RFC 2045), i cannot recollect exactly what they are now, need to dig up my old notes. So there is a System property in SAAJ  "saaj.mime.optimization" which has value "true" by default but can be set to "false" to switch-off Boyer-Moore and fallback to the old JavaMail Mime-Boundary parsing implementation. Ever since, we did have a few users report issues with the Boyer-Moore implementation and we fixed them.

SAAJ RI in its very early days was also trying to eagerly load all the attachments in an incoming message. We changed this behavior in SAAJ 1.3.2 timeframe but provided a way to fall back to the eager loading scheme incase someone needs it. For this one can set the system property "saaj.lazy.mime.optimization" to "false".  This optimization has limited utility in the absence of MimePull especially when seen in the context of  SOAPMessage.getAttachments()  and SOAPMessage.getAttachments(...) apis.  However it does help a scenario where an intermediate (SOAP Handler) recives a large MimeMessage but never needs to look at the Attachments (but only needs to read/manipulate the SOAP Envelope/Body) before resending the message to an ultimate recipient.

One property  which is only available in the latest builds  (saaj1.3.4.zip) is the system property   saaj.mime.multipart.ignoremissingendboundary  its default value is true and needs to be set to false to indicate that the SAAJ runtime should report an error if the Multipart Mime Package has a missing end boundary. This feature was requested by an end user recently on the lines of what is supported in Java Mail. Note that this feature does not work when one enables saaj.use.mimepull .   

Some Other Features of the SAAJ RI

The SAAJ RI makes use of a ParserPool (a pool of SAXParser's) and this provides upto 15% reduction in time taken when 1000's of messages are being parsed. There is a JAXP issue which prevents SAAJ RI from cleanly using a Pool of Parsers, and a workaround for this has been currently used in SAAJ RI 1.3.4. The issue was reported by one of the SAAJ users and more details are in Issue 46.  The Old implementation of the ParserPool itself was inefficient and was upgraded in 1.3.4 to make use of the BlockingQueue from the java.util.concurrent pacakge. This fix was provided by an external user (kevinconaway) as a patch.

The SAAJ RI has had support for handling incoming messages that are  Fast-Infoset encoded since SAAJ 1.3.2 and there is support for handling MTOM encoded messages as well.  Both MTOM with SOAP 1.1 and SOAP 1.2 are supported. There was an issue in MTOM handling and again that was fixed in SAAJ RI 1.3.4, a patch was provided by an external user (jeremyhuiskamp). This is primarily to support the SOAPHandlers in JAXWS 2.X. There is no way to actually create FI or MTOM encoded messages using the SAAJ API's directly.

There was an issue of poor performance with large payloads (Issue 42) and this was due to a Xalan issue (JAXP issue 48). People reading this post should make sure that they either use the steps mentioned in Issue 42 to solve this problem or get the right version of JDK Update release that has the fix for  JAXP Issue 48.

Comments

AttachmentPart custom mime-header problem

Hi All, I have a problem, Cannot set custom mime header in attachment part.

public boolean handleMessage(SOAPMessageContext messageContext) {
SOAPMessage msg = messageContext.getMessage();
// adding data sources to messages
for(String attId:_attIds){
DataSource ds = handleData.get(attId);
AttachmentPart attPart = msg.createAttachmentPart(new DataHandler(ds));
attPart.setMimeHeader("Content-Description",ds.getName());
attPart.setContentId(attId);

msg.addAttachmentPart(attPart);

}
messageContext.setMessage(msg);
return true;
}
The line :
attPart.setMimeHeader("Content-Description",ds.getName());

doesn't work, there is no attachment Content-Description in final soap message.
Can somebody help?