TOTD #81: How to use nginx to load balance a cluster of GlassFish Gem ?

nginx is known for it's high performance and low resource consumption.
It's a fairly popular front-end HTTP server in the href="http://rubyonrails.org">Rails community
along with Apache, Lighttpd, and others. This TOTD ( style="font-weight: bold;">Tip style="font-weight: bold;">Of style="font-weight: bold;">The style="font-weight: bold;">Day) will show how
to install/configure nginx for load-balancing/front-ending a cluster of
Rails application running on href="http://rubyforge.org/projects/glassfishgem/">GlassFish
Gem.

1. Download, build, and install nginx using the simple script ( href="http://snippets.dzone.com/posts/show/5668">borrowed
   from dzone):
   
   
   
   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   ~/tools
   > curl -L -O http://sysoev.ru/nginx/nginx-0.6.36.tar.gz
   
   ~/tools > tar -xzf nginx-0.6.36.tar.gz
   
   ~/tools > curl -L -O
   http://downloads.sourceforge.net/pcre/pcre-7.7.tar.gz
   
   ~/tools > tar -xzf pcre-7.7.tar.gz
   
   ~/tools/nginx-0.6.36 > ./configure
   --prefix=/usr/local/nginx --sbin-path=/usr/sbin --with-debug
   --with-http_ssl_module --with-pcre=../pcre-7.7
   
   ~/tools/nginx-0.6.36 > make
   
   ~/tools/nginx-0.6.36 > sudo make install
   
   ~/tools/nginx-0.6.36 > which nginx
   
   /usr/sbin/nginx

   
   
   OK, nginx is now roaring and can be verified by visiting
   "http://localhost" as shown below:
   
   
   
   alt=""
   src="http://blogs.sun.com/arungupta/resource/ror/nginx-welcome-page.png">
   

2. Create a simple Rails scaffold as:
   
   
   
   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   ~/samples/jruby > style="font-weight: bold;">~/tools/jruby/bin/jruby -S rails
   runner
   
   ~/samples/jruby/runner >~/tools/jruby/bin/jruby
   script/generate scaffold runlog miles:float minutes:integer
   
   ~/samples/jruby/runner >sed
   s/'adapter: sqlite3'/'adapter: jdbcsqlite3'/
   <config/database.yml >config/database.yml.new
   
   ~/samples/jruby/runner >mv
   config/database.yml.new config/database.yml
   
   ~/samples/jruby/runner >~/tools/jruby/bin/jruby
   -S rake db:migrate
3. Run this application using GlassFish Gem on 3 separate
   ports as:
   
   
   
   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   ~/samples/jruby/runner > style="font-weight: bold;">~/tools/jruby/bin/jruby -S
   glassfish
   
   Starting GlassFish server at: 192.168.1.145:3000 in development
   environment...
   
   Writing log messages to:
   /Users/arungupta/samples/jruby/runner/log/development.log.
   
   Press Ctrl+C to stop.

   
   
   The default port is 3000. Start the seond one by explicitly specifying
   the port using "-p" option ..
   
   
   

   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   ~/samples/jruby/runner > style="font-weight: bold;">~/tools/jruby/bin/jruby -S
   glassfish -p 3001
   
   Starting GlassFish server at: 192.168.1.145:3001 in development
   environment...
   
   Writing log messages to:
   /Users/arungupta/samples/jruby/runner/log/development.log.
   
   Press Ctrl+C to stop.

   
   
   and the last one on 3002 port ...
   
   
   

   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   ~/samples/jruby/runner > style="font-weight: bold;">~/tools/jruby/bin/jruby -S
   glassfish -p 3002
   
   Starting GlassFish server at: 192.168.1.145:3002 in development
   environment...
   
   Writing log messages to:
   /Users/arungupta/samples/jruby/runner/log/development.log.
   
   Press Ctrl+C to stop.

   
   
   On Solaris and Linux, you can run GlassFish as a daemon as well.

4. Nginx currently uses a href="http://wiki.nginx.org/NginxFaq#What_algorithm_does_Nginx_use_to_load_balance.3F__Can_it_balance_based_on_connection_load.3F">simple
   round-robin algorithm. Other load balancers such as href="http://github.com/gnosek/nginx-upstream-fair/tree/master">nginx-upstream-fair
   (fair proxy) and href="http://github.com/ry/nginx-ey-balancer/tree/master">nginx-ey-balancer
   (maximum connections) are also available. The
   built-in algorithm will be used for this blog. Edit
   "/usr/local/nginx/conf/nginx.conf" to specify an href="http://wiki.nginx.org/NginxHttpUpstreamModule">upstream
   module which provides load balancing:
1. Create a cluster
   definition by adding an upstream module ( href="http://wiki.nginx.org/NginxHttpUpstreamModule#upstream">configuration
   details) right before the "server" module:
   
   
   
   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   upstream
   glassfish {
   
           server
   127.0.0.1:3000;
   
           server
   127.0.0.1:3001;
   
           server
   127.0.0.1:3002;
   
       }

   
   
   The cluster specifies a bunch of GlassFish Gem instances running at the
   backend. Each server can be weighted differently as href="http://wiki.nginx.org/NginxHttpUpstreamModule#server">explained
   here. The port numbers must exactly match as those specified
   at the start up. The modified "nginx.conf" looks like:
   
   
   
   alt=""
   src="http://blogs.sun.com/arungupta/resource/ror/nginx-conf-upstream.png">
   
   
   
   The changes are highlighted on lines #35 through #39.

2. Configure load balancing by specifying this cluster using
   "proxy_pass" directive as shown below:
   
   
   
   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   proxy_pass http://glassfish;

   
   
   in the "location" module. The updated "nginx.conf" looks like:
   
   
   
   alt=""
   src="http://blogs.sun.com/arungupta/resource/ror/nginx-conf-proxy-pass.png">
   
   
   
   The change is highlighted on line #52.
   

5. Restart nginx by using the following commands:
   
   
   
   style="text-align: left; background-color: rgb(204, 204, 255); width: 100%;"
   cellpadding="2" cellspacing="2">
   sudo
   kill -15 `cat /usr/local/nginx/logs/nginx.pid`
   
   sudo nginx

Now "http://localhost" shows the default Rails page as shown below:



alt=""
src="http://blogs.sun.com/arungupta/resource/ror/nginx-rails-welcome-page.png">



"http://localhost/runlogs"
now serves the page from the deployed Rails application.



Now
lets configure logging so that the upstream server IP address and port
are printed in the log files. In "nginx.conf", uncomment "log_format"
directive and add "$upstream_addr" variable as shown: