Configuring Thread pool(s) in SailFin
SailFin SIP container requires threads to do various tasks like process an incoming request, send a response, execute timers and initiate new requests with clients. The threads required to perform these tasks are obtained from thread pools which are specifically created and configured for the sip-service in SailFin. In other words, the worker threads used in SailFin SIP container are not obtained from the legacy ORB thread pool, which a typical container inside Glassfish would do. Here is my understanding of what these thread pools are and how they can be configured, so that one can obtain the optimum performance out of the system.
The socket connector that is used by SailFin is based on Grizzly. And the Grizzly framework provides a thread pool (called as the pipeline) from which threads are obtained to process socket event (OP_READ, OP_WRITE...). Threads are obtained from this thread pool (let us call it ServersThreadPool) to parse and process the SIP request. New requests from SailFin (as UAC) get executed on the ServersThreadPool thread and any responses that are received because of requests initiated from SailFin (as UAC) would be parsed on a thread from a separate Grizzly pipeline (call it ClientsThreadPool), and executed using a thread from ServersThreadPool. RFC 3261 describes the need for transaction timers and session timers, these timer tasks are executed on the SipContainerThreadPool.
Let us look at each of the thread pools briefly and learn how they can be configured.
ServersThreadPool and ClientsThreadPool : The implementation of these thread pool lies within the Grizzly framework, and SailFin just uses this. All threads in this thread pool have to be Grizzly WorkerThread ( implement the WorkerThread interface in grizzly). When a READ event occurs on a channel, Grizzly picks a thread from this thread pool and executes the protocol filter chain on it. SailFin specific protocol filters read the data from the channel parse it and once a complete SIP message has been framed they get executed on the ServersThreadPool. The (default) Grizzly pipeline implementation is capable of queuing tasks and executing them once a free thread is available. These thread pools can be configured with maximum queue size, minimum thread count, maximum thread count, and re-size quantities.
The configuration is present in domain.XML and can be modified through asadmin set commands or the administration GUI.
initial-thread-count specifies how many threads are loaded in the pool when the pool initializes, thread-count denotes the maximum threads in the thread-pool and thread-increment will allow a user to configure the number of threads by which the pool will be expanded when there are more threads required and the number of threads in the pool is less than the thread-count.
If a user feels that 2 separate pools (pipelines) are not required and one single pipeline can be shared for the client and server tasks then that can be accomplished by setting this system property
The default behaviour (false) would be to create 2 separate pools.
The maximum number of requests that can be queued in the thread pool (pipeline) can be configured through the following property in domain.XML
A value of -1 indicates the queue is unbounded.
This property is useful when throttling is required, if the number of pending requests goes beyond the queue size then the request will be dropped and the channel closed.
The ServersThreadPool threads can be identified in the server.log using the name "SipContainer-serversWorkerThread-5060-0" where 5060 is the listener port.
The ClientThreadPool threads can be identified by the name "SipContainer-clientsWorkerThread-5060-0"
All the listeners (5060, 5061....) in the SIP container share the ServersThreadPool and ClientsThreadPool, it is per container and not per-listener. In other words, there is no partiality in executing work whether it comes from 5060, 5061 or any other sip listener, its FCFS.
SipContainerThreadPool : The implementation of this is within sailfin and each thread is a SipContainerThreadPoolThread. It is capable of queuing and executing Callable tasks on the free threads available in the pool. But the number of tasks that can be queued is unbounded (and cannot be configured) so there is no way to control/throttle the request processing in this thread pool. As of today, it shares configuration with the ServersThreadPool and ClientsThreadPool for the maximum and initial thread count values. Please follow Issue 915 for updates on this.
The timer tasks are scheduled on SIP timer queues and executed on the SipContainerThreadPoolThread. A timer queue reaper thread examines the timer queue and determines if scheduled tasks are ready to be executed, if a task is ready then it is executed on the SipContainerThreadPool. Its also possible to configure the number of timer queues (and reaper thread) on which timer tasks are scheduled.
The system property "org.jvnet.glassfish.comms.sip.timer.queues" can be used to configure the number of timer queues. By default only one timer queue is created in SailFin. If the load is high then its recommended to have more than one queue so that the timer load is distributed among them.