Is an Enterprise Service Bus (ESB) appropriate for volume-performance critical consumer applications?
In response to my SOA/ESB Level Set blog entry, I got the following question from cparziale:
"In researching esb direction, I'm trying to assess if a "service bus" is a just another service 'on the side' of the primary services, to be used when needed, like UDDI, or if it will become a physical mediator of all service consumer-provider conversations. I ask because I'm trying to visualise what to do when we have high volume-performance critical consumer applications (such as in a financial institution) who have well defined providers and don't need externalized orchestration, transformation or even routing and do not want to to incur the overhead or support of additional mediation tiers. So will the ESB concept become mediation-integration fabric, like DNS, or will it become an application tier that must be managed and accounted for in a queuing/performance analysis? Any thoughts in this space are appreciated
This is a great question, and it prompted me to write this blog entry to answer this and some related questions...
I believe the value proposition for an ESB drops considerably unless it is the mediator for all service consumer-provider conversations. The advantage of an ESB over point-to-point service invocations is mostly about managing services across the enterprise. If "rogue applications" bypass the service bus, you will lose much of its value.
At the heart of cparziale's question is the gnawing concern:
"We do not want to incur the overhead or support of additional mediation tiers."
This is obviously a legitimate concern, and one that every ESB architect must address.
What additional overhead (latency) is incurred by invoking a service via an ESB versus invoking it directly?
There are many factors to consider beyond this concern to determine whether or not an ESB is appropriate for your company, but we must still answer this question as best we can.
The answer will depend on the architecture of the specific ESB, but I believe that it is quite possible that the additional overhead could be quite low.
An ESB allows you to abstract the address of the service endpoint that you wish to invoke. To make this work, the ESB has to map the logical address of the service to a â€œphysicalâ€ address. Presumably, a service registry (akin to UDDI) will be used to look up the â€œphysicalâ€ address. Looking up the address each time a specific consumer accesses a service could be costly, but if the consumerâ€™s ESB "agent" caches "physical" addresses the overhead could be quite lowâ€¦ it all depends on the implementation of the ESB itself (Check out the links to various ESB implementations at the end of this blog entry).
Tha value of an ESB comes into play when you consider the burden that would be placed on individual service consumers if the ESB was not there... so we have to look at the benefits that the ESB provides and decide whether or not we need them.
One benefit of an ESB is the aforementioned abstraction of the "physical" address. If the location of a service moves, or if a new version of a service becomes available, the consumers of the service will not have to be modified. You could of course build this logic into the service consumer itself, but imagine where that leads.
An ESB could also be a mechanism for providing scalability and load balancing. Assume that overall performance could be enhanced by instantiating multiple copies of a specific service: The ESB could function as a load balancer, hiding from service consumers any knowledge that multiple service instances exist. Of course you could accomplish the same thing by hosting a service on a clustered server, but the ESB offers the promise of creating service instances "on demand" if coupled with the appropriate provisioning software.
Some ESB implementation provide these capabilities, but others certainly do not, and it is very important for us to ask these questions before commiting ourselves. If "high volume-performance critical consumer applications" are a part of our enterprise, then we'd better be very sure that the ESB that we adopt is up to the task... Ask the right questions early, or we could end up very disappointed.