They'll Need a Crane

How to scale your huge application

Sometimes forgotten in discussions of the scalability of applications is what the application developer brings to the table. We hear a lot about whether certain languages or app servers scale up to enterprise class, and frankly, these conversations sound a lot like performance bickering at the smaller scale. And the reason to make that analogy is that time and again, coding practices are massively underestimated as a means of improving performance. Want to run faster? Don't do stupid things like while (...) { myString = myString + anotherString } when there are more performant alternatives (StringBuffer in this pathological example). Similarly, the way you write your program has a lot to do with how it will scale.

Imagine you're performing a series of tasks for each request, and that you process hundreds or even thousands of requests each second. If you have a synchronous path through the steps required to service each request, then you're probably going to tie up one thread for each request, start to finish. Anytime any of these steps requires contending for resources or waiting for something on the network, like db query results, the thread will sit there, not doing anything. If there's a request getting rejected for lack of available threads in the pool, well, too bad for him or her.

Now imagine a more asynchronous approach: when you need to wait for something, use a messaging model and relinquish control. While one task waits for its data, another can get some CPU time. It's not the easiest thing to imagine if you don't already think multithreaded, but in the big picture -- and scalability is all about big pictures -- it increases the overall load your app can handle and improves the handling of requests.