Can Grizzly improve AsyncWeb's scalability?

A couple of months ago I've blogged about how to run AsyncWeb on top of GlassFish. Since then, Grizzly has been released as a standalone framework and it is now possible to run AsyncWeb directly on top of Grizzly.

Three simple steps:

• Install AsyncWeb 0.8.2. You can get the bundle here. Unzip it.
• Get the Grizzly framework and its asyncWeb extension here. This is Grizzly 1.0.4. I will soon push the extension to a Maven repo.
• Unzip it on top of your AsyncWeb installation. This will add two new jars under the lib directory (grizzly-framework.jar and asyncweb-grizzly­0.9.1.jar) and a new Ant file called grizzly.xml
• Invoke ant -f grizzly.xml run-grizzly to start AsyncWeb or ant -f grizzly.xml run-grizzly-examples to start the examples

Et voila!

Now just for fun, I did ran a very simple benchmark to make sure the performance of AsyncWeb is not reduced when running on top of Grizzly. I did run a couple of ab cycles by doing:

      % ab -q -n1000 -c300 http://192.168.0.105:9012/helloWorldHttpService

for AsyncWeb running on MINA and

      % ab -q -n1000 -c300 http://192.168.0.105:8080/helloWorldHttpService

for AsyncWeb running on Grizzly.

For MINA, I've increased the ioWorkerCount to 10 in asyncWeb/conf/AsynWeb.xml to get the best configuration (I've might not have found the best one...well I think I have). For Grizzly, I didn't changed anything, which means 5 threads were used.

The result is interesting. For MINA:

Server Software:        AsyncWeb
Server Hostname:        192.168.0.105
Server Port:            9012

Document Path:          /helloWorldHttpService
Document Length:        267 bytes

Concurrency Level:      300
Time taken for tests:   4.400861 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      477424 bytes
HTML transferred:       300642 bytes
Requests per second:    227.23 [#/sec] (mean)
Time per request:       1320.258 [ms] (mean)
Time per request:       4.401 [ms] (mean, across all concurrent requests)
Transfer rate:          105.89 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   67 397.6     12    3002
Processing:   137 1052 250.7   1067    1612
Waiting:      136  618 229.3    623    1227
Total:        158 1119 482.0   1078    4234

Percentage of the requests served within a certain time (ms)
  50%   1078
  66%   1228
  75%   1322
  80%   1323
  90%   1338
  95%   1344
  98%   1610
  99%   4186
 100%   4234 (longest request)

For Grizzly:

Server Software:        AsyncWeb
Server Hostname:        192.168.0.105
Server Port:            8080

Document Path:          /helloWorldHttpService
Document Length:        267 bytes

Concurrency Level:      300
Time taken for tests:   3.922259 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      424848 bytes
HTML transferred:       267534 bytes
Requests per second:    254.96 [#/sec] (mean)
Time per request:       1176.678 [ms] (mean)
Time per request:       3.922 [ms] (mean, across all concurrent requests)
Transfer rate:          105.55 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    8  12.3      0      33
Processing:    28  952 393.9   1056    1950
Waiting:       26  948 391.7   1053    1881
Total:         51  961 383.7   1056    1950

Percentage of the requests served within a certain time (ms)
  50%   1056
  66%   1180
  75%   1280
  80%   1300
  90%   1324
  95%   1364
  98%   1446
  99%   1618
 100%   1950 (longest request)

It seems AsyncWeb running on Grizzly perform a little better, but I wouldn't trust the ab result to decide to switch to Grizzly(ab is far from a good benchmark). I've used ab here just to gives some data, but If I had to make a choice, I would benchmark my application using both and get the best one.

Now the reason I was interested about getting some numbers is since the AsyncWeb integration on top of Grizzly creates a couple of Adapter classes, I suspect the performance can be improved by removing those adapters and improve the AsyncWeb|Grizzly integration. Well, that's a lot of work knowing that AsyncWeb is closely tigh to MINA (which is pretty good...don't get me wrong!). And having an alternative is always good!

Already running AsyncWeb in production? I would be interested to see if Grizzly can improve scalability or not. Feel free to ping me about your results. Since AsyncWeb can run in GlassFish, being able to deploy an AsyncWeb service inside a web application is something I want to explore, but first I need to see how it goes on top of Grizzly. I consider the integration code 'beta' and would not be surprised to discover some bugs. And maybe nobody is interested, which is something I would understand as well :-)

technorati: grizzly AsyncWeb MINA glassfish