I always find it interesting to talk to people who are engaged in developing highly-scalable software. I very much appreciate the difficulties they face and must overcome, having worked on developing multithreaded applications myself since the early 1990s (using Sun's early C SMP threading libraries).

So, I very much enjoyed talking with Sun's Owen Kellett, a lead developer for Project Darkstar, an open source, highly scalable, server-side platform for massive online games and virtual worlds. If scalable server-side platforms interest you as much as they interest me, I think you'll find the 13-minute podcast we recorded at this year's JavaOne well worthy of a listen.

Owen talks about the differences between gaming and applications such as banking as being related primarily to the trade-off between latency and data durability. For example, in a banking application, it is absolutely critical that the transaction is guaranteed to complete, or it must be rolled-back if completing it does not occur for some reason. In this situation, it's OK if there is some latency. It doesn't matter if a bank application user sometimes has to wait 25 seconds, as long as they are sure that the monetary transaction is correctly completed.

With online gaming, however, a delay of even a few seconds between when a player issues a command and sees the response can make the game unplayable. So, with online gaming and virtual worlds, minimizing latency is critical. Meanwhile, the absolute rigorous data integrity that is required in a banking application isn't necessary in an online game. Does it really matter if the last few seconds of a game player's actions are lost once every few months? Not really.

This difference is in large measure the defining factor for why a gaming-centric application server platform is needed. Most commercial application servers are tuned for the business environment, emphasizing transactional integrity at the cost of increased latency.

Still, Darkstar is a transactional system, with automated data persistence. If a server goes down, the user can resume the game normally with less than a few seconds of lost data. The Darkstar event-driven transactions are normally processed in less than a few milliseconds, much faster than is typically the case with business transactions where ACID (Atomicity, Consistency, Isolation, Durability) must be rigorously enacted. In online games and virtual worlds, it's advantageous to sacrifice a little bit on the durability side in order to dramatically decrease latency. This is what the Darkstar server accomplishes.

Project Darkstar is written entirely in Java on the server side. However, it is language-agnostic on the client side, providing the possibility for anyone to develop an API that can interact with the Darkstar server. The external developer community has already developed Flash clients, C# clients, Python clients, and others.

The Darkstar server currently scales superbly when it is run on a single machine with multi-core processors. The server can run on multiple nodes as well. Currently, significant effort is being put into increasing the scalability in the multi-node environment, where you face the issues of having multiple nodes interacting with a single data store on the back end.

In addition, the team is researching means to automatically discover when multiple clients are interacting with each other, modifying the same data areas. In this case, in the multi-node environment, it is most efficient to have the same node perform the processing for both clients.

I asked Owen about "the 15-minute experience" I'd seen on the web site. What's this? Basically, the Darkstar server can be downloaded, installed, and deployed in 15 minutes. You can get your first little demo application (provided with the download) running in just 15 minutes, which confirms for you that you have a proper install and base configuration.

If you're interested in online gaming and virtual worlds software, and/or scalable transactional platforms in general, I think you'll find listening to Project Darkstar: the Open Source Highly-Scalable Server-Side Gaming Platform an interesting way to spend 13 minutes and 34 seconds (yes, in the podcast Owen beat his own "15-minute experience" criterion!).