Skip to main content

Introduction to Grizzly-Thrift and sharing the benchmarking results

Posted by carryel on December 21, 2011 at 12:43 AM PST

This page is for introducing Grizzly-Thrift server/client modules and sharing various benchmarking results.

Object serialization/deserialization of Java comes expensive. For improving this lack, we sometimes used to use other frameworks for RPC such as Protobuf and Thrift which support various programming languages, RPC and own data structures.

Especilally, Thrift has already provided various transport types. Basically, there are TSimpleServer, TThreadPoolServer, TNonblockingServer and THsHaServer for server and there is TSocket for client.

But Thrift's transport layer can be replaced for performance improvement by other NIO frameworks. So I tried to make another transports based on Grizzly and benchmark it experimentally. It's Grizzly-Thrift server/client module.

Grizzly framework is for building scalable and robust servers using NIO and also offering extended framework components: Web Framework (HTTP/S), Bayeux Protocol, Servlet, HttpService OSGi and Comet.

Therefore it was not difficult for me to support Thrift server/client using Grizzly.

- Grizzly-Thrift Server/Client Modules -

Grizzly-Thrift included in Grizzly version 2.2 which released at 2011/12/20 but it was moved into different repository after Grizzly v2.2.2.

You can review and download sources as the following.

<a href="http://java.net/projects/grizzly/sources/thrift/show">http://java.net/projects/grizzly/sources/thrift/show</a>

git://java.net/grizzly~thrift (read-only)

For using Grizzly-Thrift server, you should add ThriftFrameFilter and ThriftServerFilter to Grizzly transport. For using Grizzly-Thrift client, you should add ThriftFrameFilter and ThriftClientFilter to Grizzly transport.

ThriftFrameFilter encodes/decodes Thrift's TFramedTransport which is composed of frame-length header(4bytes) and body. And ThriftServerFilter/ThriftClientFilter interconnects the user's processor/handler with TGrizzlyServerTransport/TGrizzlyClientTransport which extends Thrift's TTransport.

Here are examples for Thrift server/client based on Grizzly.

--- Grizzly-Thrift Server ---

      <pre>
final FilterChainBuilder serverFilterChainBuilder = FilterChainBuilder.stateless();

final user-generated.thrift.Processor tprocessor = new user-generated.thrift.Processor(new user-generated.thrift.Handler());

serverFilterChainBuilder.add(new TransportFilter()).add(<strong>new ThriftFrameFilter()</strong>).add(<strong>new ThriftServerFilter(tprocessor)</strong>);

final TCPNIOTransport grizzlyTransport = TCPNIOTransportBuilder.newInstance().build();

grizzlyTransport.setProcessor(serverFilterChainBuilder.build());

grizzlyTransport.bind(port);

grizzlyTransport.start();

 

--- Grizzly-Thrift Client ---

          <pre>
final FilterChainBuilder clientFilterChainBuilder = FilterChainBuilder.stateless();

clientFilterChainBuilder.add(new TransportFilter()).add(<strong>new ThriftFrameFilter()</strong>).add(<strong>new ThriftClientFilter()</strong>);

final TCPNIOTransport grizzlyTransport = TCPNIOTransportBuilder.newInstance().build();

grizzlyTransport.setProcessor(clientFilterChainBuilder.build());

grizzlyTransport.start();

final Future&lt;Connection&gt; future = grizzlyTransport.connection(ip, port);

final Connection connection = future.get(10, TimeUnit.SECONDS);

final TTransport tGrizzlyTransport = <strong>TGrizzlyClientTransport.create(connection)</strong>;

final TProtocol tprotocol = new TBinaryProtocol(tGrizzlyTransport); // or TCompactProtocol

user-generated.thrift.client.Client client = new user-generated.thrift.Client(tprotocol);

// ... user specific client call

If you are already familiar to Thrift, this is easy. If you aren't, I recommend that you review Thrift's tutorial examples first. (See the JavaServer.java and JavaClient.java in Thrift's tutorial.)

Grizzly-Thrift modules already include basic unit tests based on Thrift's tutorial. (See the ThriftTutorialTest.java in Grizzly-Thrift.)

If you are using maven in your project, here are pom.xml's dependencies.

--- pom.xml ---

<pre>
...

&lt;dependency&gt;

    &lt;groupId&gt;org.glassfish.grizzly&lt;/groupId&gt;

    &lt;artifactId&gt;grizzly-framework&lt;/artifactId&gt;

    &lt;version&gt;2.2.3&lt;/version&gt;

&lt;/dependency&gt;

&lt;dependency&gt;

    &lt;groupId&gt;org.glassfish.grizzly&lt;/groupId&gt;

    &lt;artifactId&gt;grizzly-thrift&lt;/artifactId&gt;

    &lt;version&gt;1.0&lt;/version&gt;

&lt;/dependency&gt;

...

In addition, Grizzly provides various IO strategies such as worker-thread, same-thread and leader-follower so I also tried to test Grizzly-Thrift modules with each IO strategy. If you would like to know more Grizzly's IO strategies, please see the this.

- Benchmarking -

I also benchmarked various Thrift Server-Client modules which are TSocketServer/Client, TThreadpoolServer, TTNonblockingServer, Netty Server/Client and Grizzly Server/Client. I used business operations based on Thrift's tutorial for test but modified a bit logic for packet size.

Test Information

  • Server Type/Client Type: TServer-TSocketClient vs TServer-NettyClient vs TServer-GrizzlyClient vs GrizzlyServer-TSocketClient vs GrizlzyServer-GrizzlyClient vs etc...
  • Message Size: About 3M Bytes, 3K Bytes, 300 Bytes
  • Thrift Protocol: Binary, Compact
  • Client Connections: 20~1000
  • Server and Client Test Machine Information
  • Scenario
    • After 1min warming-up, testing 5min and collecting total results

Benchmarking Results

  • 3M + Compact + 40 Connections
    Server Types TSocket Client Netty Client Grizzly Client
    TServer 8,637 478 8,510
    TThreadPoolServer 11,221 2,273 11,220
    TNonblockingServer 11,223 1,832 11,221
    Netty 11,220 2,311 11,220
    Grizzly 11,221 1,765 11,225
    • Netty client had the performance issue unfortunately, so I would exclude it for next benchmarking.
  • 3M + Binary + 40 Connections
    Server Types TSocket Client Grizzly Client
    TThreadPoolServer 11,219 11,215
    TNonblockingServer 11,221 11,221
    Netty 11,213 11,221
    Grizzly 11,220 11,222

In 3M test, Compact/Binary and Server/Client tests were meaningless with regards to performance.

  • 3K + Compact + 40 Connections
    Server Types Grizzly Client
    TThreadPoolServer 8,283,705
    TNonblockingServer 5,801,319
    Netty 9,058,550
    Grizzly 8,964,358
    Grizzly(SameIO) 9,098,590
    • TNonblockingServer had the performance issue. And Netty and Grizzlys' results were better than Thrift server modules'.
  • 3K + Binary + 40 Connections
    Server Types TSocket Client Grizzly Client
    TThreadPoolServer 7,619,693 8,163,692
    TNonblockingServer 5,444,630 6,032,290
    Netty 8,254,168 8,930,896
    Grizzly 8,204,097 8,833,978
    Grizzly(SameIO) 8,257,918 8,960,497
    • Grizzly client module had better performance than TSocket client so I would use only Grizzly client for next benchmarking.

In 3K test, Compact protocol was better than Binary protocol. And Netty and Grizzlys' results were better than Thrift server modules' so I would use only Netty and Grizzly server for next benchmarking.

  • 300Bytes + Compact + 20 Connections
    Server Types Grizzly Client
    Netty 10,269,876
    Grizzly(SameIO) 10,349,440
    Grizzly(LeaderF) 9,654,216
  • 300Bytes + Compact + 40 Connections
    Server Types Grizzly Client
    Netty 14,569,820
    Grizzly(SameIO) 14,770,452
    Grizzly(LeaderF) 13,674,641
  • 300Bytes + Compact + 60 Connections
    Server Types Grizzly Client
    Netty 15,783,774
    Grizzly(SameIO) 15,962,425
    Grizzly(LeaderF) 15,227,426
  • 300Bytes + Compact + 80 Connections
    Server Types Grizzly Client
    Netty 16,964,578
    Grizzly(SameIO) 16,712,315
    Grizzly(Worker) 15,890,537
    Grizzly(LeaderF) 16,252,280
  • 300Bytes + Compact + 100 Connections
    Server Types Grizzly Client
    Netty 15,879,803
    Grizzly(SameIO) 15,781,153
    Grizzly(Worker) 16,136,977
    Grizzly(LeaderF) 16,437,650
  • 300Bytes + Compact + 120 Connections
    Server Types Grizzly Client
    Netty 15,904,968
    Grizzly(SameIO) 15,985,106
    Grizzly(Worker) 16,097,609
    Grizzly(LeaderF) 16,164,636
  • 300Bytes + Compact + 150 Connections
    Server Types Grizzly Client
    Netty 15,952,442
    Grizzly(SameIO) 16,109,154
    Grizzly(Worker) 16,261,584
    Grizzly(LeaderF) 15,923,040
  • 300Bytes + Compact + 500 Connections
    Server Types Grizzly Client
    Netty 12,463,442
    Grizzly(SameIO) 12,499,963
    Grizzly(Worker) 12,461,131
    Grizzly(LeaderF) 12,532,517
  • 300Bytes + Compact + 1000 Connections
    Server Types Grizzly Client
    Netty 11,867,630
    Grizzly(SameIO) 11,903,400
    Grizzly(Worker) 11,906,507
    Grizzly(LeaderF) 11,812,262

In many connections(more than 120 connections), most of servers didn't receive proper requests of client because the client machine of this environment used too much resouces such as high CPU usages. So I think that more client machines are needed to calculate meaningful data of more connections. In 100 connections, Netty and Grizzly's same-thread IO strategy's throughput decreased but Grizzly's woker-thread IO and leader-follower IO strategies' throughput increased.

In my test cases and environments, worker-thread IO strategy and leader-follower IO strategy were more effective than same-thread IO strategy if servers should have more than 100 connections.

- Conclusion -

 

  • Results of 300Bytes + Compact + 40 Connections
    Server Types TSocket Client Netty Client Grizzly Client
    TServer 741,417   604,558
    TThreadPoolServer 14,731,560   12,747,230
    TNonblockingServer 6,060,111   6,723,402
    Netty 14,749,519   14,569,820
    Grizzly(SameIO) 14,931,745 9,066,525 14,770,452
  • Results of 3KBytes + Compact + 40 Connections
    Server Types TSocket Client Netty Client Grizzly Client
    TServer 631,300   526,341
    TThreadPoolServer 7,708,088   8,283,705
    TNonblockingServer 5,264,995   5,801,319
    Netty 8,372,804   9,058,550
    Grizzly(SameIO) 8,381,352 3,718,431 9,098,590
  • 300Bytes + Compact + 100 Connections
    Server Types Grizzly Client
    Netty 15,879,803
    Grizzly(SameIO) 15,781,153
    Grizzly(Worker) 16,136,977
    Grizzly(LeaderF) 16,437,650
  • Server Module
    • Grizzly same-thread IO strategy was best in a few connections. Grizzly leader-follower IO strategy was best in many connections.
    • CPU Usages: Netty==GrizzlySameIO < GrizzlyLeaderFollowerIO < GrizzlyWorkerIO
  • Client Module
    • In small packets, TSocket was best. In larget packets, Grizzly client was best.
  • Thrift Protocol
    • In this scenario, Compact protocol was best.

Finally, I decided that our company, Kakao would use worker-thread IO strategy for Grizzly-Thrift server in real fields because it was very stable. For client I decided that I would use same-thread IO Strategy because of efficiency.

If you are already using Thrift or have a plan to use Thrift for RPC, just try to apply Grizzly-Thrift!

(I think these results are only for reference so you could meet different results according to your environments and benchmarking logic).

Related Topics >>
Blog Links >>