Skip to main content

Introducing Dalma Project

Posted by kohsuke on October 12, 2005 at 8:57 PM PDT

Many applications of today need to have a part of the program that waits for other entities. For example, a mailing list manager waits for an e-mail to trigger actions. A web application waits for incoming HTTP requests.

Sometimes those applications are conversational; they not only just receive messages, but also send out messages. Those programs often maintain some state and change it as the conversation progresses. Often there are multiple conversations running concurrently. Those are what we call "workflow" applications.

For example, where I work, I have a little process every Monday. On Sunday night, cron triggers an automated weekly build of our software. Upon a successful build, it sends out an e-mail. On Monday morning, a manager and an engineer replies to those e-mails, authorizing the posting of the build to a publicly visible web server. Another program waits for those authorization e-mails, push the binaries to the server, and then report the result in an e-mail, concluding a conversation. This little ritual happens concurrently for each project in my group.

Today, those applications can be written, but one can't write it very productively. For one thing, every time an incoming message comes, the program has to retrieve the state information, act on it, and then store it back. This needs a lot of boilerplate code. You also need to write it as an event-driven program, where your application consists of a set of message handlers. This makes it difficult to follow the flow of the logic.

Dalma makes it very easy to write those workflow applications by letting you write it as an ordinary procedural program without any boilerplate.

For example, consider the following program that subscribes an user to a mailing list by using a simple e-mail base confirmation:

EmailEndPoint eep = ...;

void subscribe(MailingList ml, InternetAddress userEmailAddress) {
  // ask the user to confirm subscription
  MimeMessage msg = new MimeMessage();
  msg.setText("are you sure you want to subscribe?");
  msg.setRecipient(TO,userEmailAddress);
 
  // send it and then wait for the reply
  msg = eep.waitForResponse(msg,1,WEEK);
 
  // if the user acknowledges ...
  if(msg!=null)
      // then subscribe
      ml.add(userEmailAddress);
  else
    // if no response is received in a week,
    // abort.
    ;

While this is very easy to write, traditionally, writing such a program like this would be considered a bad idea, because:

  1. this requires one thread per one conversation, making it less scalable
  2. if the JVM shuts down, all information about the on-going conversations will be lost.

Dalma solves those issues by allowing you to run them on a "virtual thread". A virtual thread is like an ordinary thread, except that it can be suspended, persisted, and resumed later, possibly in a different JVM. For those of you who have some experience in Scheme, this is similar to call/cc.

Inside the waitForResponse method, dalma suspends the virtual thread, and then persists its execution state to a disk. The actual Java thread that was used to run this virtual thread can be then reused to run other virtual threads. Later, when an e-mail is received, the persisted execution state is restored, and the virtual thread resumes its execution inside the waitForResponse method. The JVM can even shut down and restart between this.

This mechanism eliminates the needs of the explicit state persistence; those states are now captured in terms of local variables and call stacks. Dalma engine handles persistence transparently behind the scene. This mechanism also allows few threads to support much larger number of virtual threads, which is essential for supporting many concurrent conversations.

The call/cc part of the functionality is available as Apache Jakarta commons Javaflow project. The workflow engine part is available from the dalma project here at java.net. Both are still work in progress, but let me know what you think.

Related Topics >>