Tricks and Tips with NIO part III: To Thread or Not to Thread
Posted by jfarcand on July 07, 2006 at 04:42 AM | Comments (6)
This time I will share some observations I've experimented when handling OP_ACCEPT, OP_READ and OP_WRITE using Threads. When I started working on Grizzly, I've designed the framework open enough so I can easily add thread pool mostly everywhere during the request processing. At that time there weren't a lot of NIO framework available neither clear recommendations about what to do and what to avoid. To avoid having to redesign Grizzly every weeks, I've decided to make OP_READ, OP_ACCEPT and OP_WRITE processing configurable. By configurable, I mean being able to execute different strategies, e.g. being able to execute the processing of those operations on their own thread or using the same thread as the Selector:
if ( myExecutor == null ){
myExecutor = Executors.newFixedThreadPool(maxThreads);
}
try{
selectorState = selector.select(selectorTimeout);
} catch (CancelledKeyException ex){
;
}
readyKeys = selector.selectedKeys();
iterator = readyKeys.iterator();
while (iterator.hasNext()) {
key = iterator.next();
iterator.remove();
if (key.isValid()) {
if ((key.readyOps() & SelectionKey.OP_ACCEPT)
== SelectionKey.OP_ACCEPT){
myExecutor.execute(getAcceptHandler(key));
} else if ((key.readyOps() & SelectionKey.OP_READ)
== SelectionKey.OP_READ) {
myExecutor.execute(getReadHandler(key));
}
....
} else {
cancelKey(key);
}
}
From the code above, the getAcceptHandler(key) will return a Runnable object which most usually does:
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel = server.accept();
if (channel != null) {
channel.configureBlocking(false);
SelectionKey readKey =
channel.register(selector, SelectionKey.OP_READ);
setSocketOptions(((SocketChannel)readKey.channel()).socket());
}
where getReadHandler(key) will do:
key.interestOps(key.interestOps() & (~SelectionKey.OP_READ));
socketChannel = (SocketChannel)key.channel();
while ( socketChannel.isOpen() && (
((count = socketChannel.read(byteBuffer))> -1)){
// Do something
An alternative is to execute the getAcceptHandler(key)and getReadHandler(key) on the same thread as the Selector.select(..):
try{
selectorState = selector.select(selectorTimeout);
} catch (CancelledKeyException ex){
;
}
readyKeys = selector.selectedKeys();
iterator = readyKeys.iterator();
while (iterator.hasNext()) {
key = iterator.next();
iterator.remove();
if (key.isValid()) {
if ((key.readyOps() & SelectionKey.OP_ACCEPT)
== SelectionKey.OP_ACCEPT){
getAcceptHandler(key);
} else if ((key.readyOps() & SelectionKey.OP_READ)
== SelectionKey.OP_READ) {
getReadHandler(key);
}
....
} else {
cancelKey(key);
}
}
and of course, without having to create a Runnable object. Finally, the other alternative is a mix of the first two strategy:
try{
selectorState = selector.select(selectorTimeout);
} catch (CancelledKeyException ex){
;
}
readyKeys = selector.selectedKeys();
iterator = readyKeys.iterator();
while (iterator.hasNext()) {
key = iterator.next();
iterator.remove();
if (key.isValid()) {
if ((key.readyOps() & SelectionKey.OP_ACCEPT)
== SelectionKey.OP_ACCEPT){
getAcceptHandler(key);
} else if ((key.readyOps() & SelectionKey.OP_READ)
== SelectionKey.OP_READ) {
myExecutor.execute(getReadHandler(key));
}
....
} else {
cancelKey(key);
}
}
or executing the getAcceptHandler(key) using myExecutor and
getReadHandler on the same thread as the Selector.select(..). Like I said earlier, Grizzly can be configured to support all strategies.
Which strategy perform the best
I've benchmarked all of the above strategies and find that the one that perform the best is:
try{
selectorState = selector.select(selectorTimeout);
} catch (CancelledKeyException ex){
;
}
readyKeys = selector.selectedKeys();
iterator = readyKeys.iterator();
while (iterator.hasNext()) {
key = iterator.next();
iterator.remove();
if (key.isValid()) {
if ((key.readyOps() & SelectionKey.OP_ACCEPT)
== SelectionKey.OP_ACCEPT){
getAcceptHandler(key);
} else if ((key.readyOps() & SelectionKey.OP_READ)
== SelectionKey.OP_READ) {
myExecutor.execute(getReadHandler(key));
}
....
} else {
cancelKey(key);
}
}
c'est a dire executing the OP_ACCEPT on the same thread as the Selector.select(..), and using a Thread for executing the OP_READ. I've shared my observations with my colleagues and they also came to the same conclusion.
Well, what about OP_WRITE
I didn't forget the OP_WRITE. I also tested the strategies described above and came to the conclusion than OP_WRITE should be handled using the same Thread as the one handling OP_READ. One thing that might explain why I'm getting such results is the use of temporary Selector when the main Selection is not able to flush the socket outgoing buffer (see part I for more details), or when socketChannel.read() return 0. Another important observation is all the tests I've ran are either HTTP or IIOP based protocol. Other protocols might perform differently, although I suspect it will not make such a difference.
Are you getting different results? This is not easy to measure because you have to make sure the framework itself is not the bottleneck.
As usual, feedback is more than welcome. Next time I will discuss using more than one Selector under high load. Merci!!
technorati: grizzly nio glassfish
Bookmark blog post:  del.icio.us  Digg  DZone  Furl  Reddit
Comments
Comments are listed in date ascending order (oldest first) | Post Comment
-
Have you thought at all about potentially merging some of your work with MINA so that we can make a single high-quality NIO protocol toolkit? We'd love to have your contributions!
Posted by: peterroyal on July 07, 2006 at 05:18 AM
-
What do you mean by "...perform the best..."?
Pushing the most data out to fast clients?
Pushing the most data out to slow clients?
handling the largest number of clients?
serving the most numbers of requests?
If you have many slow clients that write half a http header before you have to wait for more data I do think that your "fastest" version will block with all worker threads tied up waiting for data.
If I remember correctly you can select the strategies in grizzly, so perhaps that is not a big problem in your case.
Performance is tricky.
Posted by: ernimril on July 07, 2006 at 03:00 PM
-
Hi Peter, I've just started learning Mina via AsyncWeb, which we are trying to make it work on top of Grizzly. Once this exercise is done, I will know for sure if Grizzly and what I'm discussing here can be implemented using Mina. I wil drop some emails on your mailling list :-)
Posted by: jfarcand on July 10, 2006 at 07:27 AM
-
By "...perform the best...", I mean pushing the most data to medium/fast client and handling the largest number of clients (measuring both simultaneously). I agree "Performance is tricky.", but my recommendations doesn't necessarely means holding a thread if the data aren't coming. You can always register back the SelectionKey to the Selector thread and free the worker thread. You are rigth about Grizzly: this is configurable........If someone has some data on slow network, I would really like to see them. Thanks!
Posted by: jfarcand on July 10, 2006 at 07:41 AM
-
What about a LeaderFollower Threading model?
public class NIOConnection(){
ByteBuffer readBuffer;
ByteBuffer writeBuffer;
AtomicInt isRunning = new AtomicInt(0);
}
public class LeaderFollowerWorker implements Runnable{
Iterator iterator;
Selector selector;
Executor executor;
ConcurrentLinkedQueue keyList = new ConcurrentLinkedQueue();
public LeaderFollowerWorker(Selector selector, Executor executor){
this.selector = selector;
iterator = selector.selectedKeys().iterator();
this.executor = executor;
}
public void run(){
try {
while (!iterator.hasNext()){
selector.select();
registerOldKeys();
iterator = selector.selectedKeys().iterator();
}
SelectionKey = iterator.next();
iterator.remove();
if(key.isAcceptable())
doAccept(key);
else
doRead(key);
} catch (Exeption e){
e.printStackTrace();
} finally {
executor.execute(this);
}
doSomething(key); // Do the hardwork
NIOConnection conn = (NIOConnection)key.attachement();
int isRunning = conn.getIsRunning().getAndSet(0);
if (isRunning == 2) {
keyList.offer(key);
selector.wakeup();
}
}
private void doRead(SelectionKey key){
NIOConnection conn = (NIOConnection)key.attachement();
int isRunning = conn.getIsRunning().incrementAndGet();
if(isRunning == 1){
// byteBuffer = ThreadLocal readByteBuffer.get();
while(sc.isOpen() && (sc.read(byteBuffer)) > 0);
conn.setReadBuffer(byteBuffer);
} else // isRunning == 2
key.interestOps(key.interestOps() & (~SelectionKey.OP_READ));
}
// Accept, create the Connection object and register
private void doAccept(SelectionKey key){
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel;
// Accept all the pending clients
while ((channel = server.accept()) != null) {
channel.configureBlocking(false);
SelectionKey readKey =
channel.register(selector, SelectionKey.OP_READ, new NIOConnection());
setSocketOptions(((SocketChannel)readKey.channel()).socket());
}
}
private void registerOldKeys(){
SelectionKey key;
while((key = keyList.poll()) != null){
try {
key.interestOps(key.interestOps() & SelectionKey.OP_READ);
} catch (Exception) { e.printStackTrace();}
}
}
}
public void static Main(String[] args){
Selector selector = ... // open the Selecor and register the ServerSocketChannel
Executor executor = Executors.newFixedThreadPool(maxThreads);
for(int i=0; NumberOfCPUs >= i; i++){
executor.execute(new LeaderFollowerWorker(selector, executor);
}
}
Built by Text2Html
Posted by: vbeltran on July 11, 2006 at 08:05 AM
-
Hi Vincenç, your code isn't compilling ;-) I've tried this pattern a long time ago and didn't see any benifit. But you are rigth, I should have included the result here. I will test it based on Grizzly impl and update this thread once I have it. Thanks!
Posted by: jfarcand on July 12, 2006 at 03:59 PM
| 
java.net RSS Feeds
