Skip to main content

Top Java 5 EE Servers Compared

Posted by jdcampbell on February 20, 2008 at 9:23 PM PST

size=4>Top JEE 5 Comparison of Features and Benchmarks.


Do you need to make a good decision on what Java EE application
server to use for your organization? Sure, and there are a lot of choices out
their. I have used Java application servers for years my self, and total there
are actually about 9 different JEE application servers on the market that I know
of. Open-source Java EE servers are used as much as the commercial counterparts
and I believe the only advantage of going commercial is that you may get better
security that way. Though certain commercial JEE servers have proprietary
features which are valuable, the same can be said of open-source
available features.


The servers we will look at are Sun Microsystems open-source
Glassfish 2.0, Red Hat’s open-source JBoss 4.2, IBM’s open-source WebSphere 2.0
Community Edition, and BEA’s commercial WebLogic 10. I'm covering JBoss 4.2 due to
it's popularity, though it is only partially JEE 5 compliant. I will do benchmarks using a standard benchmark tool from WiseFirm.com. All of these
servers and the benchmarks are being run on the same computer and on the same
Linux OS, so the benchmarks are a very good demonstration of how these servers
truly compare on performance.

Features Comparison…

Below is a quick table on the major features of importance
available with our four JEE servers, these features are the standards used in the industry. The table shows a "Yes" if the feature is shipped with the JEE server package standardly, the table shows "Available" if the feature requires additional package installs, and the "Partial" marks a feature that the particular JEE server does not fully support...


style="BORDER-RIGHT: #0000cc 0px solid; BORDER-TOP: #0000cc 0px solid; BORDER-LEFT: #0000cc 0px solid; BORDER-BOTTOM: #0000cc 0px solid"
borderColor=#0000cc cellSpacing=0 borderColorDark=#0000cc cellPadding=0
borderColorLight=#0000cc border=1 align="center">


Feature Description

Glassfish 2.1




JBoss 4.x




WebSphere 2.0




WebLogic 10





Java EE 5 compliance

Yes




Partial




Yes




Yes





JSP 2.1 and Servlet 2.5 capable

Yes




Yes




Yes




Yes





EJB 3.0 capable

Yes




Yes




Yes




Yes





JavaServer Faces 1.2 support

Yes




Yes




Yes




Yes





Custom plug-in support

Yes




Yes




Yes




Yes





Business-rules engine support

Yes




Available




Available




Available





Hibernate 3.x support

Available




Yes




Available




Available





JBoss Seam support

Available




Yes




Yes




Available





Clustering support

Yes




Yes




Yes




Yes





JAX-WS / JAX-B 2.x

Yes




Available




Yes




Yes





Eclipse IDE connector support

Yes




Yes




Yes




Yes





Ease of operation rating 1 to 10




6




9




6






Benchmark Comparison...

I used a
really nice tool for benchmarking called jRealBench (www.WiseFirm.com gives it
free). This tool is designed to show realistic benchmarks, it uses a simple Java
.war application that you deploy on your J2EE server, then you specify how many
session hits you want to benchmark and just click the Benchmark button. It keeps
a database of all the computers you benchmark and allows you to do a chart
comparison of them. Here are the results of our four servers all run on the same
x64 Dual-core server, this benchmark is based on a 10,000 session test...




There are two
tests done in the benchmark charts above, the left test is initial session
creation which is when a user first hits your web site application, the right
side bars are session re-hit benchmarks which is when the same user accesses
another page in your web application. Just in case you have any trouble reading
the server names, the blue-bar is JBoss, the green-bar is WebSphere, the first
red-bar is Glassfish, and the right-hand red-bar is Weblogic.


This test
does not simply check the maximum number of sessions that can be run on these
J2EE servers, this is a more realistic benchmark comparable to a production web
application. A previous article I did showed that many J2EE servers will run
over 1.2 million concurrent sessions, and that is only limited by RAM
availability on the server from my knowledge.


Many people
have stated that Glassfish is faster then these other servers, but I ran these
benchmarks again and again and could not get Glassfish to outperform the others.
As you can see Glassfish benchmarked slower then 2 of the other servers, and
Weblogic was the slowest of all. WebSphere 2 CE is the same as Geronimo 2, it is
based on the Tomcat 6 servlet container, and clearly outperforms all of these
other J2EE servers. WebSphere (Geronimo) is the fastest J2EE server I know
of.

The Bottom Line...

Saving money and getting the best performance and product for
the buck is what we all need. Websphere 2.0 CE (Geronimo) is supported by at
least as many people as JBoss 4.x is, and Websphere outperforms all of these
other J2EE servers. Red Hat's JBoss and BEA Weblogic have commercial support,
while Websphere CE and Glassfish are primarily open-source products. You will
pay more for the commercial support but rather you end up in better shoes for
that is your decision.



As far as
J2EE version 5 compatibility and features are concerned, all of these servers
are JEE 5 compatible except JBoss 4.x. Red Hat is releasing JBoss 5 soon and
that will be JEE 5 compliant and probably perform better, but I don't think it
will perform better then Websphere/Geronimo noticeably, though we will just have
to see. On the matter of features overall, the most popular JEE components
fortunately are very cross-platform compatible so most servers support them.

Related Topics >>

Comments

If you do not want your article to be criticized, why bother with them? Even better, why enable comments? Just disable it

JD Campbell

In Memorium: My brother, Jonathan Campbell, hung himself in Febuary, 2009. With all his faults, I still miss him. I wish him well. L Campbell

My goodness. RIP.

My goodness. RIP.

Simple: All you critics need to give the address of your blogs or professional articles! Thats it. Put your money where your mouth is. Maybe you guys are just Microsoft zombie critics, thats the situation!!!

I worked on Unix systems in about 1987 with my first job, and I have been a tech ever since. I think this blog is excellent. I have done tests and what JVM you use makes vary minor difference on the performance.

Also see Scott's Note on NIO vs IO. - eduard/o

Are you running WebLogic in development mode instead of production mode? I know that in development mode, the server checks to see if the code has been modified (more user friendly) VS production mode where none of these checks cause any latency.

"Here are the results of our four servers all run on the same x64 Dual-core server..." _WHERE_ are the results? they're invisible? looking at the html source of this page shows that you intended to display this image after the above sentence: http://www.inetvibe.com/cmsfiles/JEE_Bench_Chart.jpg but its a dead link. what a wasted of time!

Is there any benchmark done against the WebSphere 6.X Version. That would be really helpful for me. Cheers

It amazes me how you and other people can criticize, but you give no useful information your self. Sure, there are other things that should be benchmarked on JEE servers, but this is just a Blog, I dont get paid for this. And if a JEE server benchmarks poorly on the foundation features (Servlets) then it indicates that other things are as well, slow. Do your own tests, post a blog, then you will have the right to judge my benchmarks! My certifications are in Java, Databases, Linux, AIX, Solaris, MS Windows server, and many more. My college certification is in Java, Linux, and MS Windows server. I am working on a degree now. Education is what it's all about.

"I have 15 tech certifications and college, what do you have?" Now, that's really mature response to a critique. Well, we have doubts that benchmarking Application Servers only by their Servlet Container capabilities is worth any attention. By the way, "Ease of operation rating 1 to 10" is bollocks. You should at least provide explanations what 1, 2, 3, etc. stands for.

I'll advise you to test with the best JVM that each server supports. For instance, Glassfish runs on the latest and greatest Sun JDK 6u4, which may not be not true for all other servers. WebLogic should really be tested on JRockit, etc. This is a very important issue because Java EE app servers are highly dependent on the base Java SE runtime - more so than even the most complex standalone Java SE apps... that's why all major vendors have their own JVM, so they can tune it do death in a way that favors their appserver - they perform integrated optimization of the entire stack. (The "real" WebSphere can't even run on Sun VMs, at least not with some massive effort, because it depends on lots of IBM JDK-specific stuff like CORBA and security). Vendors who put the effort to have their own JVM, or at least to guarantee compatibility with the latest JVMs, deserve the extra points delivered by these. Vendors who have stupid compatibility bugs, like many past releases of JBoss that didn't like the JMX of JDK 6, deserve the perf disadvantage of the two-years-old JVM technology that they support. Also, try a real benchmark. I looked briefly at jRealBench, and it's definitely in the "toy" category (for your purpose anyway). The benchmark code fits in a single 46Kb jar file, that should be enough. It sems to use an embedded Sleepycat database for storage, also not a very serious way to measure real-world Java EE app performance. Adding insult to injury, the benchmark code is obfuscated and no source code is available - I'll trust a benchmark that doesn't disclose source code the day hell freezes over.

I apologize, I have been rude and that wasn't my intention. Obviously you are posting this with the best of intentions, and I should never write after a long day. Sorry. My comment sounded hard because I have worked with all these servers (but geronimo) since weblogic 5 and jboss 2.2, until today. I miss the advanced features of these servers from this comparison, for example a "cluster: yes" checkmark leaves out all the cluster JDBC features that weblogic provides. Same with the amazing Web Services stack that comes with glassfish. A JMS features comparison would also be welcome, or some performance comparisons between web/JMS/web services components. Sorry for being offensive, I didn't mean it.

Mr. Icoloma, tell me where your blog is? I would like to see what you know that makes you such an authority to down my Blog. I have 15 tech certifications and college, what do you have? And JBoss does not have the same score as WebLogic, if you see clearly JBoss clearly performs faster in the chart then Weblogic.

You are seriously posting this? The same score to JBoss 4 and Weblogic 10? Performance based on the 10% of a real application requirements? No mention to JMS, WS, or even plain web tier... This is wrong on so many levels...

Answers: I used the Sun 1.5 JVM on all tests. I spoke with WebLogic and they said the standard configuration should give good benchmarks? Thats what they said? Concerning IBM partner benchmarking tools: the jRealBench tool does the identical benchmark for all the J2EE servers, it has no partiality for any JEE server. It does very standard Servlet/session operations, so any JEE server should give good performance for Servlet/session testing. I basically used standard configuration on all of these JEE servers. You can download jRealBench from www.Wisefirm.com and do the tests your self on any JEE server, jRealBench allows anyone to do benchmarks with their own configuration and their own hardware choice, unlike SPECj which is a test done by each JEE server maker historically and so the hardware used for the test is chosen by the JEE server maker. I spoke directly with some of the Glassfish team and they said Glassfish should perform well with standard configuration? Concerning the Ease-of-operation: Those 1 to 10 ratings are based on ease of deploying a Java application, installation, and user management, etc.

hey Jonathan: What JVM was used for this test? For example, WebLogic claims that by using JRockit combined with WebLogic... you get the fastest server in the market... And based on your tests... it looks like WL is the slowest... So I'm wondering if the tests were run with Sun's JVM underneath WebLogic? Can you provide us with some details on the JVM that was used in each case? (including the version) Thanks Venkatt

Websphere, the product IBM's consultants sells you, which is in its version 6.1 and supports J2EE 1.4, is just a completely different thing than Websphere Community Edition, the IBM-branded Geronimo, which is in its version 2.0.0.1 and supports Java EE 5.
I know you say in your text that it is the CE version, but using just 'Websphere' in the charts is misleading. At least put a 'CE' in them :)

How does web layer can be deterministic factor for application servers performance level. Application servers are not only web layer server, but they are mostly business operation and transaction management servers. So please refer to SpecJ benchmarks even if available results refer to SpecJ 1.4 instead of letting web layer determine these servers performance level

Even if your benchmark reflect anything, your post does not contain enough information in order to reproduce the test, I use WebLogic and I know how does it perform when you configure it in an appreciate way. Although I am not referring to HTTP Session management.

When you install GlassFish it installs in Developer profile which as the name shows it is a profile suitable for developers. not for any kind of benchmarking. So please post your Glassfish configuration for more clarification

Another item is your "Ease of Operating" factor, Can you explain how you come to this conclusion that GlassFish rank is 7 and WebLogic rank is 6 out of 10?

Can you post which routine tasks are included in your "Ease of Operation" test?

Which tasks you have performed on all servers, how much steps each taks requires and how much complex those steps are?

Your post has no indication that you count one of the most important aspect of application servers which is horizontal scalability aka Clustering and load balancing along with your performance benchmark. Let us know how "Ease of Operation" rank changes when we need to set-up a cluster of several nodes and perform routine tasks on the cluster.

Nice post after your article in javaWorld

PS: The title looks odd

Hrm, it does look like a pretty graph BUT how do I reproduce /test your results - you didn't provide any system specifics, nor any tuning settings for each application server. You may want to change the word compliance to compatible. WebSphere 2.0 is not compliant where as GlassFish v1 and v2 is Java EE 5 compliant. The use of an *IBM* partner testing tool showing the *IBM* product to be 'superior' sounds a little too "convenient" to me... An application that creates X number of sessions, recorded in X amount of time is hardly a JEE application... Have you looked at testing these products using a well known spec.org benchmark - SPECjAppServer2004, posting your configuration settings etc, for example: http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-2007... I see on IBM;'s website that it does not support high availability or failover, nor does it provide a centralised administration for clustering... How exactly did you rate administration Console/etc? Can you please detail your requirements for a 1, let alone 10...? OK that will do for now, Chris

Sigh.. I think I'll let somebody else grill you on this post. - eduard/o