Skip to main content

Introducing mimepull - Streaming API for MIME messages

Posted by jitu on October 12, 2007 at 3:54 PM PDT



New project mimepull
allows applications to access MIME attachments in a streaming
fashion.  Often times,  larger MIME messages cannot
be loaded
into
memory. Hence the whole message or attachment parts are written to a
file system and access to the attachment parts is provided using those
files.

But in some cases, the MIME attachment parts can be accessed
by
applications in a streaming fashion, provided:

  • The parts are accessed
    orderly(as they appear in the stream)
  • The parts are accessed only
    once.

In such situations, the parts need not be written to file
system(however large message it is !). How is it implemented
? MIME
message parsing is done using pull-parsing, much similar to href="http://www.jcp.org/en/jsr/detail?id=173">StAX
in XML world. Using the parser one can pull chunk after chunk of the
MIME message. 

for e.g.:<br>public void parse() {<br>    MIMEParser parser = new MIMEParser(InputStream, boundary, ...);<br>    Iterator&lt;MIMEEvent&gt; it = parser.iterator();<br>    while(it.hasNext()) {<br>        MIMEEvent event = it.next();<br>        switch(event.getEventType()) {<br>            case START_MESSAGE :<br>            case START_PART :<br>            case HEADERS :<br>            case CONTENT : // called multiple times for a attachment<br>            case END_PART :<br>            case END_MESSAGE :<br>       }<br>    }<br> }<br>

But an application developer doesn't have to worry about this
low-level parsing details. There is a nice high-level API to get
different attachment parts in a streaming fashion. The
MIMEParts are constructed lazily, and parsing is
triggered by applications while reading the attachment parts.
MIMEConfig provides various configuration options to control parsing
and storing MIME parts. It is also possible to read MIME parts in any
order and multiple times, but doing so may create attachment parts on
the file system.

// Parts are accessed in order. The data is accessed in a streaming fashion<br>// using readOnce() and data is not stored in temp files.<br>public void parse() {<br>    InputStream in = ...<br>    String boundary = ...<br>    MIMEConfig config = ...<br>    MIMEMessage mm = new MIMEMessage(in, boundary, config);<br><br>    MIMEPart partA = mm.getPart("partA");<br>    InputStream ais = partA.readOnce();<br>    // read ais<br><br>    MIMEPart partB = mm.getPart("partB");<br>    InputStream bis = partB.readOnce();<br>    // read bis<br><br>}&nbsp;

The implementation would be easier if the parts are accessed
in an order. Since parts can be accessed in any order, it required more
careful approach.  Also while the first part's stream is read
half way, the application could access the second part. We wanted to
have an efficient implementation and avoid writing to file system
whereever we can(some situations, we cannot avoid writing those large
MIME parts to temp files). Thanks to href="http://weblogs.java.net/blog/kohsuke/">Kohsuke
who helped me in implementing this in an efficient way.

In a future blog, I would write how mimepull is used in JAX-WS
RI 2.1.3 and JAX-WS RI applications take advantage of this to
send/receive larger attachments.


Blog Links >>