John Reynolds's Blog

Community: Java Web Services and XML Archives

The Anatomy of a Human Powered (web) Service

Human Powered Services (HPS) are the piece of the puzzle that meshes BPM with SOA... With HPS in the mix choreographing Human and Autonomous services is seamless, without HPS in the mix choreographing Human and Autonomous services is a pain.

Continue reading at my other blog...

Human-Powered Web Services

As SOA has become a more mainstream IT concept... you might even say a more mundane IT concept... there has been a lot of thought put forth on the creation of Composite Applications.... collections of distinct Services that are orchestrated and choreographed together (using BPM technologies) to implement some "higher-level" business functionality.

Many of these Composite Applications need user-interfaces, just like "normal" applications, but fortunately the transition from building a UI on top of objects (often EJBs) to building a UI on top of Services (often Web Services) is generally not that hard for programmers to make. Perhaps the biggest adjustment is in moving from a world where pass-by-reference was possible to a message-oriented world where everything is pass-by-value... but that's not really much of an obstacle for good programmers.

A more subtle aspect of user-interfaces and Composite Applications becomes apparent when you start to consider the Services that make up the applications. Most of the folks that I talk with tend to think of Services as automatons... fully automated tasks carried out by machines. This is especially true when I use the term Web Services... Everybody knows that a Web Service has nothing to do with people, it's just a machine at the "other" end of the line.

I'm going to suggest that conceptually limiting Web Services to automatons is a mistake that has hampered our ability to benefit from all that SOA has to offer. Our shared belief that Web Services are "always" automatons has impacted our conception of Composite Applications and has even impacted the design of most of our development environments (IDEs).

That's a pretty strong statement, so I'd better try to explain myself. Perhaps an example is in order....

Let's say that you are building a Composite Application to process an Order of some sort. Also assume that your customers speak many languages, and that it's necessary to translate each order to a common language before it is processed. To make this process pretty darn simple, let's also assume that Services are available to translate the orders from one language to another. Piece of cake... just funnel the order through the Language Translation Service and you're done.

Any good Service-Oriented IDE (such as NetBeans with the Enterprise Pack ) can handle the above process with ease. The IDE's tools make it relatively simple to orchestrate the process using BPEL, and relatively simple to build a UI on top of the BPEL (it's a Web Service after all). In short-order, you've created your Composite Application and all is well in the world.

So where's the beef? What's wrong with this picture?

Nothing really... until you start to consider the creation of the Language Translation Service with your IDE. Our conception of Web Services as automatons causes few problems when consuming the Services, the problem becomes evident when creating them.

I picked Language Translation on purpose... It's hard. Go out to any of the free translation services, translate this page to another language and back again, and you will see what I mean. Language Translation (at this point in time) is better performed by a person than by a machine.

Here's the previous paragraph translated to Spanish and back to English:

Try building a Human-Powered Web Service with your IDE. You can do it, but it's tough. You're going to have to cobble together something that links together a human-oriented application with a Web Service, and it's probably not going to be pretty. This "ugliness" really shouldn't be a big surprise, because we've never thought of doing such a thing (maybe "never" is a bit strong... but not very often).

Now that I am spending much more time as a Business Process Guy, I'm finding the need for Human-Powered (Web) Services isn't rare at all. All of the good BPM suites (like Lombardi and Fuego) erase most of the distinctions between creating Human-Powered Services and creating Autonomous Services because most business processes are a mix of both types of Services.

Unfortunately, at this time there aren't widely-adopted standards for Human-Powered Services (HPS)... the BPM suites are proprietary and the concept of a "stand-alone" HPS seems alien. There's a good reason for that... when an HPS is invoked, the Human has to "know" that there is work to be done and that implies a notification mechanism like a task list or email.

I don't want to imply that it will be "easy" to come up with standards for Human-Powered Services... just look at all of the turmoil surrounding "BPEL for People"... but it is time for us to start addressing this class of Service in our IDEs. In this blog entry I have focused mostly on the creation of the HPS... but there are also some issues of consumption (people tend to be much slower than machines).

What it all really boils down to is wider awareness of Process-Oriented concerns... business processes are made up of automated and human-performed tasks, and we need to be able to implement both types of tasks in a manner that allows them to be consumed by a multitude of applications. That's what SOA was really all about in the first place.

Human-Powered Web Services

As SOA has become a more mainstream IT concept... you might even say a more mundane IT concept... there has been a lot of thought put forth on the creation of Composite Applications.... collections of distinct Services that are orchestrated and choreographed together (using BPM technologies) to implement some "higher-level" business functionality.

Many of these Composite Applications need user-interfaces, just like "normal" applications, but fortunately the transition from building a UI on top of objects (often EJBs) to building a UI on top of Services (often Web Services) is generally not that hard for programmers to make. Perhaps the biggest adjustment is in moving from a world where pass-by-reference was possible to a message-oriented world where everything is pass-by-value... but that's not really much of an obstacle for good programmers.

A more subtle aspect of user-interfaces and Composite Applications becomes apparent when you start to consider the Services that make up the applications. Most of the folks that I talk with tend to think of Services as automatons... fully automated tasks carried out by machines. This is especially true when I use the term Web Services... Everybody knows that a Web Service has nothing to do with people, it's just a machine at the "other" end of the line.

I'm going to suggest that conceptually limiting Web Services to automatons is a mistake that has hampered our ability to benefit from all that SOA has to offer. Our shared belief that Web Services are "always" automatons has impacted our conception of Composite Applications and has even impacted the design of most of our development environments (IDEs).

That's a pretty strong statement, so I'd better try to explain myself. Perhaps an example is in order....

Let's say that you are building a Composite Application to process an Order of some sort. Also assume that your customers speak many languages, and that it's necessary to translate each order to a common language before it is processed. To make this process pretty darn simple, let's also assume that Services are available to translate the orders from one language to another. Piece of cake... just funnel the order through the Language Translation Service and you're done.

Any good Service-Oriented IDE (such as NetBeans with the Enterprise Pack ) can handle the above process with ease. The IDE's tools make it relatively simple to orchestrate the process using BPEL, and relatively simple to build a UI on top of the BPEL (it's a Web Service after all). In short-order, you've created your Composite Application and all is well in the world.

So where's the beef? What's wrong with this picture?

Nothing really... until you start to consider the creation of the Language Translation Service with your IDE. Our conception of Web Services as automatons causes few problems when consuming the Services, the problem becomes evident when creating them.

I picked Language Translation on purpose... It's hard. Go out to any of the free translation services, translate this page to another language and back again, and you will see what I mean. Language Translation (at this point in time) is better performed by a person than by a machine.

Here's the previous paragraph translated to Spanish and back to English:

Try building a Human-Powered Web Service with your IDE. You can do it, but it's tough. You're going to have to cobble together something that links together a human-oriented application with a Web Service, and it's probably not going to be pretty. This "ugliness" really shouldn't be a big surprise, because we've never thought of doing such a thing (maybe "never" is a bit strong... but not very often).

Now that I am spending much more time as a Business Process Guy, I'm finding the need for Human-Powered (Web) Services isn't rare at all. All of the good BPM suites (like Lombardi and Fuego) erase most of the distinctions between creating Human-Powered Services and creating Autonomous Services because most business processes are a mix of both types of Services.

Unfortunately, at this time there aren't widely-adopted standards for Human-Powered Services (HPS)... the BPM suites are proprietary and the concept of a "stand-alone" HPS seems alien. There's a good reason for that... when an HPS is invoked, the Human has to "know" that there is work to be done and that implies a notification mechanism like a task list or email.

I don't want to imply that it will be "easy" to come up with standards for Human-Powered Services... just look at all of the turmoil surrounding "BPEL for People"... but it is time for us to start addressing this class of Service in our IDEs. In this blog entry I have focused mostly on the creation of the HPS... but there are also some issues of consumption (people tend to be much slower than machines).

What it all really boils down to is wider awareness of Process-Oriented concerns... business processes are made up of automated and human-performed tasks, and we need to be able to implement both types of tasks in a manner that allows them to be consumed by a multitude of applications. That's what SOA was really all about in the first place.

Business Process Musings

For the past few months I've been working with a very polished Business Process Management (BPM) suite from Lombardi Software , and I have to say that I am as happy as a clam. I feel like I have discovered a tool that I've been missing for at least 20 years, maybe longer, and I'm delighted.

It's hard to explain BPM to those who haven't experienced it, and to complicate matters it seems that each BPM vendor defines the space a bit differently, but I think that the dawning availability of high quality BPM suites is going to have an impact on business programming that will be just as significant as Object-Oriented programming was to programming at large. Quality BPM suites will usher in an era of Process Driven Design, and the wide-spread adoption of that paradigm is going to lead to a renaissance in business process engineering.

Okay, so you are now thinking that I've found another shiny penny, and that I'm vaulting back into hyper-enthusiast mode... Well... Not really. This is pretty much the same song that I've been singing for the past few years, but a different verse.

Remember SOA (Service Oriented Architecture)? Remember how folks couldn't really grok the difference between Services and CORBA and EJBs?

The answer is surprisingly simple... There really isn't much difference between Services and CORBA and EJBs in terms of functionality. The difference is in terms of "composability", and that's where BPM tools are making the difference. BPM tools give you the ability to compose (in a very agile fashion) multi-party process-oriented applications that can incorporate Services.

At this point you may be thinking that I'm describing BPEL (the Business Process Execution Language), but BPEL's only a small subset of what BPM offers (BPEL doesn't even handle human interactions yet).

BPM suites allow you to create multi-party process-oriented applications from scratch, deploy them to production, and maintain them over the long haul. You begin by diagramming the business process using a graphical notation such as the Business Process Modeling Notation (BPMN). The resultant Business Process Diagram (BPD) captures the highest level requirements in an executable process definition... This is one of the great things about BPM, the mapping between the requirement and the implementation couldn't be more straight-forward. The BPD can be stepped through with the business folks ad-infinitum to make sure that it's really correct, and together you can add in all of the (business) exception flows and caveats until everyone is happy.

Meanwhile, the activities that make up the process have to be fleshed out in detail. Some activities will require human interaction, others will be fully automated. Some activities will leverage pre-existing services, others will require new services... Good BPM suites contain tools for creating UI's, constructing light-weight services, and connecting to existing services (Web Services, Java objects, databases, packaged applications like SAP, etc.)... Most of the BPM suites that I'm familiar with are Java EE based, so what you're getting is a process-oriented veneer on top of all the Java functionality that you know and love.

Once the activities have been fleshed out and tested, the resultant application can be deployed to a Process Execution Engine (PXE) and made available to the masses... just like a Java EE application. But unlike a generic Java EE application, the PXE's generally provide a great deal of support for process-level instrumentation, and this is where Business Process Engineering comes into play.

Software Engineers generally care about infrastructure metrics. How fast is a query? What's the latency of response time for an incoming message? How much space is left on the hard drive?

Business Process Engineers deal at a different level. How long does it take to process an order? Where is the bottleneck in the process?

A good BPM suite (which includes a good PXE) allows the Business engineer to ask these questions directly. Each process can be instrumented in a natural fashion to obtain the answers that the Business engineer needs. If you've ever worked in an IT department you've probably had to scour through log files to answer a Business question that you thought they'd never ask. You've probably also had to develop some sort of ad-hoc instrumentation in your applications to capture metrics that might be useful someday, but will most likely never be evaluated.

BPM suites go a long way towards curing those headaches for you by coupling straightforward process instrumentation with focused reporting tools. Drag a few instrumentation points onto your BPD, create a simple report, and now your Business Process Engineers have the data that they need to really tune their processes.

None of what I am describing is really new, but the combination of these features into a truly usable environment is something that I hadn't experienced before, and you are probably going to have to experience it for yourself to truly grok the value. Unfortunately the Lombardi tool that I am using has no community edition, but there are some pretty good alternatives out there to sink your teeth in. I haven't had a chance to use them yet, but Intalio's Open Source BPM and JBoss's jBPM both look promising (and you can try them for free).

If you are a nuts and bolts infrastructure guy, or a game designer/simulation god, then BPM probably won't be your cup of tea... but if you're a business programmer, BPM is probably going to make your job a lot more fun.

Business Process Musings

For the past few months I've been working with a very polished Business Process Management (BPM) suite from Lombardi Software , and I have to say that I am as happy as a clam. I feel like I have discovered a tool that I've been missing for at least 20 years, maybe longer, and I'm delighted.

It's hard to explain BPM to those who haven't experienced it, and to complicate matters it seems that each BPM vendor defines the space a bit differently, but I think that the dawning availability of high quality BPM suites is going to have an impact on business programming that will be just as significant as Object-Oriented programming was to programming at large. Quality BPM suites will usher in an era of Process Driven Design, and the wide-spread adoption of that paradigm is going to lead to a renaissance in business process engineering.

Okay, so you are now thinking that I've found another shiny penny, and that I'm vaulting back into hyper-enthusiast mode... Well... Not really. This is pretty much the same song that I've been singing for the past few years, but a different verse.

Remember SOA (Service Oriented Architecture)? Remember how folks couldn't really grok the difference between Services and CORBA and EJBs?

The answer is surprisingly simple... There really isn't much difference between Services and CORBA and EJBs in terms of functionality. The difference is in terms of "composability", and that's where BPM tools are making the difference. BPM tools give you the ability to compose (in a very agile fashion) multi-party process-oriented applications that can incorporate Services.

At this point you may be thinking that I'm describing BPEL (the Business Process Execution Language), but BPEL's only a small subset of what BPM offers (BPEL doesn't even handle human interactions yet).

BPM suites allow you to create multi-party process-oriented applications from scratch, deploy them to production, and maintain them over the long haul. You begin by diagramming the business process using a graphical notation such as the Business Process Modeling Notation (BPMN). The resultant Business Process Diagram (BPD) captures the highest level requirements in an executable process definition... This is one of the great things about BPM, the mapping between the requirement and the implementation couldn't be more straight-forward. The BPD can be stepped through with the business folks ad-infinitum to make sure that it's really correct, and together you can add in all of the (business) exception flows and caveats until everyone is happy.

Meanwhile, the activities that make up the process have to be fleshed out in detail. Some activities will require human interaction, others will be fully automated. Some activities will leverage pre-existing services, others will require new services... Good BPM suites contain tools for creating UI's, constructing light-weight services, and connecting to existing services (Web Services, Java objects, databases, packaged applications like SAP, etc.)... Most of the BPM suites that I'm familiar with are Java EE based, so what you're getting is a process-oriented veneer on top of all the Java functionality that you know and love.

Once the activities have been fleshed out and tested, the resultant application can be deployed to a Process Execution Engine (PXE) and made available to the masses... just like a Java EE application. But unlike a generic Java EE application, the PXE's generally provide a great deal of support for process-level instrumentation, and this is where Business Process Engineering comes into play.

Software Engineers generally care about infrastructure metrics. How fast is a query? What's the latency of response time for an incoming message? How much space is left on the hard drive?

Business Process Engineers deal at a different level. How long does it take to process an order? Where is the bottleneck in the process?

A good BPM suite (which includes a good PXE) allows the Business engineer to ask these questions directly. Each process can be instrumented in a natural fashion to obtain the answers that the Business engineer needs. If you've ever worked in an IT department you've probably had to scour through log files to answer a Business question that you thought they'd never ask. You've probably also had to develop some sort of ad-hoc instrumentation in your applications to capture metrics that might be useful someday, but will most likely never be evaluated.

BPM suites go a long way towards curing those headaches for you by coupling straightforward process instrumentation with focused reporting tools. Drag a few instrumentation points onto your BPD, create a simple report, and now your Business Process Engineers have the data that they need to really tune their processes.

None of what I am describing is really new, but the combination of these features into a truly usable environment is something that I hadn't experienced before, and you are probably going to have to experience it for yourself to truly grok the value. Unfortunately the Lombardi tool that I am using has no community edition, but there are some pretty good alternatives out there to sink your teeth in. I haven't had a chance to use them yet, but Intalio's Open Source BPM and JBoss's jBPM both look promising (and you can try them for free).

If you are a nuts and bolts infrastructure guy, or a game designer/simulation god, then BPM probably won't be your cup of tea... but if you're a business programmer, BPM is probably going to make your job a lot more fun.

Thoughts on "The Modular JRE" and Open Sourcing Java

David Herron posted a clarification of "what it means to be Java" on his blog, and the examples that he used got me thinking...

David made it clear that anyone can add whatever they like to their version of the JRE (Java Runtime Environment) and still call it Java... but if they leave anything out then it is most certainly not Java. The example he sited was the hypothetical incusion of SWT (the windowing toolkit used by Eclipse) in a version of the JRE. This version could still label itself as Java as long as AWT and Swing (the windowing toolkits in the Java specifications) are not excluded.

This makes sense to me... but I'm hoping that the folks who are opening Java also take another look at the basic definition of what must be in the JRE. Let me explain my concern...

Many Java applications (maybe even most) have no desktop-based user interface at all (they are browser-based web applications)... and if the adoption of Web Services and SOA continue we'll see more and more "headless" Java-implemented Services (Services that have no user interface at all). In a world where it is quite legitimate to have a large percentage of Java applications with zero need for any windowing toolkit, why carry around those toolkits in the JRE?

I know that code that isn't used doesn't really "cost" much, but let's assume that you are really into Grid computing, and you have thousands of nodes in your cluster... and each node has a JRE with unused windowing toolkits on it. Seems wasteful.

Windowing toolkits in the base JRE are fairly easy to throw stones at, but there are many other components that really should be optional. For an example, just take a look at the recent "discussions" on whether or not JavaDB should be included in the Java Developer Kit (I shudder to think of the threads that would be spawned if JavaDB is ever packed into the JRE).

My impression is that there is a large consensus that the JRE should be more modular, broken into logical chunks that are installed based on need. A modular JRE does pose some rather interesting technical challenges (to get the code that you need when you need it), but I had overlooked the challenge posed by the current definition of Java itself.

If I'm reading David right... a modular JRE wouldn't quite meet the current definition for "Java"... it's all or none. If I never install the "windowing-toolkit" chunk on my application servers, then they aren't really "Java" application servers.

Just something to think about...

Is the End of Tiered-Based Computing in Sight?

This morning I came across a whitepaper from GigaSpaces entitled: Space-Based Architecture and The End of Tier-based Computing

Perhaps the most widely adopted style of software architecture is the N-Tier architecture... the separation of concerns based on stacked tiers of functionality. The "Three-Tier" architecture is perhaps the best known N-Tier approach, with functionality separated into Presentation Logic, Business Logic, and Data Access Logic.

The premise put forward by GigaSpaces is that N-Tier solutions are hitting a wall in terms of scalablity, or more specifically that the increasingly complicated schemes necessary to scale an N-Tier solution are hitting a roadblock.

The basic approach to scaling an N-Tier solution is to deploy multiple instances of any Tier that is having trouble meeting the necessary performance goals. For example, if the EJBs in your Business tier can't keep up with the requests from your Presentation tier, then deploy copies of the EJBs on additional servers.

This approach to scaling an N-Tier application works pretty well unless you need to deploy many copies of your Business tier... the middleware necessary to deploy the EJBs and to load balance requests becomes increasingly complicated, not to mention the overhead incurred when sending messages to remote tiers.

GigaSpace suggests a different approach to scaling, which they have christened Space-Based Architecture:

"Space-Based Architecture (SBA) represents a new model that combines distributed caching ("Data Grid"), content-based messaging ("Messaging Grid") and parallel processing ("Processing Grid") for transforming existing tier-based applications into linear, dynamically scalable services. These new middleware components are implemented on top of a distributed shared memory space that provides common clustering, high availability, location transparency and consistency services across all tiers."

Radically simplifying SBA (you really ought to read the whitepaper), the gist seems to be that tiers are not deployed separately. The tiers required to process the application logic are grouped into a single logical processing unit, and scaling is achieved by running multiple instances of those units on multiple machines. To contrast this with the typical approach to scaling an N-Tier application, instead of creating multiple instances of the tiers, the approach is to create multiple instances of the application:

"The power of spaces comes from not having to separate various parts of an application into discrete physical runtimes — and then wiring those together in complex, hard-to-scale, and performance-consuming tangles of middleware. A space doesn’t care if an application has been “tiered.” Whether it has or not, the same program code will instantiate multiple times on the same machine or on multiple machines automatically — and even dynamically - in response to runtime parameters like CPU utilization." - Nati Shalom - CTO, GigaSpaces Technologies

So what do you think? Is Space Based Architecture and the emergence of Grids and Service-Oriented Architecture going to change the way most software architects approach their designs... or is this just a niche that won't make much of an impact?

Disconnected Browser-Centric Clients

There is a huge paradigm gap between writing a desktop application with something like Swing, and writing a browser-based application with something like Java Server Faces. Frameworks like NextApp's Echo2 go a long way towards narrowing the gap, but there are still underlying disconnects that just won't go away...

Just from the practical standpoint of learning how to program, this is a big deal.

I know that some folks wish that browser-based applications would disappear... but I think that adopting that credo would be a mistake. The "old school" desktop applications aren't truly suited to a mobile lifestyle where an individual may connect to the Web from a variety of devices. In a single day I might use my home computer, PDA, mobile phone, work computer, and public kiosk to access my applications and data.

That said, I must admit that most of the current crop of browser-based applications don't quite cut the mustard: If you can't connect to the Web, then you can't use your applications.

We need a blend... We need applications that can be accessed from any device connected to the Web, but with persistent components that reside on the devices that we actually own.

I'll use Google's Calendar to illustrate what I have in mind: I love having a calendar that I can access from any web connection (actually, it doesn't work from my PDA yet, but hopefully they are working on that). I also love the ability to share my calendar with others (particularly with my wife).

What I don't love is losing access to my appointments when I am not online.

I want a persistent component of my calendar on all of the devices that I own. I want my devices to "synchronize with the mother ship" when connected to the Web, updating themselves from a central data store, and passing along any appointments that I might have recorded while offline.

Fortunately, Google has thoughtfully provided a Web Service Interface to their Calendar, so it is quite possible to create a client application to satisfy my wants and needs... I can have a desktop application in addition to my browser-based application if I so choose.

The down side to dual applications is that there's something clunky about this approach...
Accessing a desktop application when not online, and a browser-based application when I am online seems rather like a kludge.

Call me crazy, but what I'd like would be to always start from my browser, type in or select a URL, and have the appropriate version of the application run... If I am online, give me access to the full blown web application. If I am offline, give me access to the persistent components on my local machine. Adam Bosworth has written extensively on the subject of disconnected web applications... but his solutions always seem to end up relying on making changes to the browsers themselves, and I just don't think that's likely given the vested interests of browser manufacturers.

Roshan Shrestha's article "Take Your Tomcat On The Road" gave me an idea for supporting disconnected web applications that does not require the support of browser manufacturers: Create a locally hosted web application to do your bidding.

In my Calendar example, write a locally hosted web application that interacts with the Google Calendar web service. To access this application, simply enter the proper URL in the browser. If the device is offline, the local web application displays the "disconnected client" interface... If the device is online, the local web application redirects the user to the "online client" interface provided by Google.

I might be the only person on the planet who finds this approach attractive... but then again maybe there are a lot of other odd people out there who want something similar.

The biggest obstacle that I see to this approach is the installation of the "disconnected client" on the local machine. Roshan has demonstrated that it's a breeze to bundle a fully functional web application in a neat little package... but how to get that package onto a user's devices?

Would it be possible to use Java Web Start to install something like Roshan's "Tomcat On The Road" package on a user's machine? I'm really not sure... but hopefully somebody out there can help me find out.

Continue Reading...

Objects are Nouns... Services are Verbs... SOA hinges on designing the right Verbs

I still find postings that express confusion about the relationship between object-oriented architectures and service-oriented architectures... so I would again like to offer my own $0.02 worth of insight...

Objects are Nouns; Services are Verbs

An Object is primarily a Thing that is manipulated (and is generally stateful), a Service is primarily an Action that is performed (and is generally stateless).

In an object-oriented architecture, the software components are primarily organized as entities (things) that are to be manipulated. These entities generally have relationships to other entities, and that leads to the emphasis on design artifacts such as UML class diagrams.

In a service-oriented architecture, the software components are primarily organized as tasks (actions) that are to be performed. These tasks are generally designed to function as steps in more complex processes, and that leads to the emphasis on design artifacts such as BPMN diagrams.

Businesses use software to automate business processes. Business processes do involve Nouns (Objects), but they are generally more concerned with Verbs (Services).

Let me explain: Most businesses process forms of one sort or another. In general, manipulation of the form (an object) is not the primary concern of the business; the primary concern of the business is to perform a set of tasks (services) based on the contents of the form.

If the software components are packaged as distinct services, the mapping between the business requirement to "process the form" and the software implementation of the process is relatively straight forward. This clean mapping between requirements and implementation results in solutions that can better respond to changing requirements over time (Trust me, all business requirements change over time).

Sun's Tim Bray has recently expressed his opinion that the term "SOA" is meaningless, and that he prefers the term "Web-style". With respect to Tim, I disagree with his opinion.

I do not agree with Tim that "Web-style" is a better term than "SOA".

I agree with Tim that the term SOA has:"become damaged, weakened by over hype, over use, over promise (and) under deliver", but unlike Tim, I think that I can "explain what the difference is between SOA and Web services". Web services are a protocol specific form of service. To implement an SOA you must eventually pick a protocol, but it needed be a Web oriented protocol.

Tim asks for "an explanation that's meaningful in terms a programmer can understand, so a programmer can go and build one of these things". Perhaps it is my own arrogance or naivete, but I think my explanation is a darn good start.

In my experiences with many developers, I have found that explaining how to design and build SOA solutions is not any more difficult or "verbose" than it was to explain how to design and build Object-Oriented solutions. I confess that I do resort to "paragraphs and paragraphs (of) prose", but everyone who knows me understands that I always resort to paragraphs and paragraphs of prose ;-)

But if Tim is right, and the term SOA has become a dead alabatross, then I'd like to offer an alternative to "Web-style": How about Process Oriented Architecture? Kind of has a nice ring to it; Don't you think?

Regardless of the acronym or term that we use to describe it, the point remains the same: If your primary task is to automate a business process, then you should package (most of) your software components as verbs... er, I mean services.

Ken Orr's advice to Java programmers (and the secrets of writing good software)

In the 1970's, Ken Orr expanded on Jean-Dominique Warnier's methodology to develop Data Structured System Design (DSSD).

DSSD is primarilly a data driven methodology for designing systems with (gasp!) direct user involvement. Through the use logic and data charts that are easily explained to the users, the requirements of a project are collaboratively captured in a format from which much of the code and data schemas can be generated... This may sound like "no big deal" today, but Ken published these techniques in his book Structured Systems Development in 1977.

Ever the opportunist, I cornered Ken during lunch one day to pump him for insights (as fodder for this blog)...

John: "Ken... What advice would you give Java programmers?"

Ken Orr: "Don't define yourself as Java programmers."

Being a good programmer really has very little to do with the languages that you know... Being a good programmer is mostly dependent on the methodologies that you know and on how you employ those methodologies.

Good process design skills and good data modelling skills are the foundations for being a good programmer.

Process design and data modelling are best done in collaboration with your users (Don't confuse this with letting the users design the process or model the data, both require considerable skills that aren't easily mastered). Ken offered this advice when developing a new design with a client:

It is better to be clearly wrong than to be obscurely right.

If you have something wrong in your design, and it is clearly wrong, then your client will point it out and you can correct it. If the design is obscure, the client may never know if it is right or if it is wrong.

Tools are the key for communicating with your clients... processes (and data) are expressed well through pictures (flow charts, etc.), and you are missing the boat if you don't adopt tools that map these pictures to code.

I got the feeling that Ken's biggest regret is that Computer Aided Software Engineering (CASE) seems to have been on hiatus for most of the 90's and the first years of this century. Model Driven Architecture is surely a form of CASE, but Ken feels that the focus isn't quite right.

UML was developed to meet the design needs of programmers. We need tools focussed on meeting the design needs of users.

This led us into a discussion of SOA and BPEL. Ken is very "hype-sensitive", and cautions that SOA must be understood before you take the plunge.

One of the most promising aspects of SOA is that it brings the art of process design back into focus. In many cases, each service can serve as a step in a process, and BPEL can be used to describe the process itself.

Ken's parting advice was perhaps the most important thing to remember...

Master more than one design/programming paradigm.

Ken gave the example of AOP vs. OOP. Objects are very good at solving "vertical" problems... Hierarchies of relationships being the most common. Aspects are very good at solving "horizontal" problems... Cross-cutting concerns require thinking beyond object boundaries.

I suspect this is why Ken has never become too old to program. By mastering the basics of data modeling and process design, and by mastering newer paradigms as they are developed, Ken has remained a vibrant force in the programming world.

I don't want to make Ken blush, but in my book he's a darn good role model.

Continue Reading...

Service Oriented Mashups

I came across this quote recently:

"What programmers in a hundred years will be looking for, most of all, is a language where you can throw together an unbelievably inefficient version 1 of a program with the least possible effort." Paul Graham

"We know we don't have a corner on creativity. There are creative people all around the world, hundreds of millions of them, and they are going to think of things to do with our basic platform that we didn't think of." Vint Cerf

Perhaps that is true for the moment, but the climate for "business oriented" mashups is about to get a lot more interesting...

AJAX does a fair job of cobbling services together on a web page, but what about cobbling together services into long running processes? Tools for orchestrating services have been around for a few years, but just as it took Google Maps to propel XmlHttpRequest into the limelight, we haven't really seen Service Orchestration take off yet.

Free orchestration tools like NeBeans 5.5 may just change that.

NetBeans 5.5 "out of the box" integrates the PXE BPEL engine and a Visual Orchestration Designer, dramatically lowering the "barrier to entry" for anyone who wants to experiment with composite service oriented applications.

I have been a proponent of Process Driven Design for quite some time, but until now it has always been quite a challenge for anyone to get started; too many parts to download and install. With NetBeans 5.5, getting started will almost be child's play... In a single (free) IDE you will be able to create Service Endpoints and Orchestrate them into long running processes (Oracle's BPEL Designer for Eclipse is similar, but has a few strings).

What this means to my fellow Service Oriented Architects is that your services are about to get a whole lot more accessible. "Power Users" within your own company are about to discover the bounty that you have provided, and just as Google never dreamed what the public would do with their maps, you are about to discover that your own services can be "mashed up" in ways that you never imagined.

Let the games begin ;-)

Continue Reading...

Service Oriented Architects

In his XML Annoynaces blog, Micah Dubinko offers his perspective on software architecture and architects:

"It goes without saying that a major development project should have a solid architect at the core. A good architect needs to be able to separate personal preferences and prejudices from legitimate good design points--in other words, get the focus right. A concrete measure of an architect is whether outside groups--even ones that wouldn't normally be directly involved--are able to understand and accept the architecture. There will be times where it's just not possible to please everyone to the level of consensus, but still it provides a measuring rod against which to evaluate a design (or a designer)."

If you work with an IT group, you (like me) are probably involved in some sort of Service Oriented Architecture discovery or implementation project; Gartner predicts that SOA will provide the basis for 80 percent of development projects by 2008.

Given this huge trend towards SOA, we really need Service Oriented Architects who can "get the focus right".

Reuse is an often mentioned benefit of SOA, but I believe that SOA's key benefit is the closer mapping between business processes and the implementation. SOA meshes perfectly with Process Driven Design.

Many Object Oriented (Jave and C++) projects have great technical architectures, but the business process implementation ends up scattered across many modules and hidden in the code. When a business process changes (and they often do) it is quite often difficult (or at least tedious) to implement and deploy the process oriented changes: Not exactly what business wants.

I don't know why this happens, but I suspect a subtle impedance mis-match between form (architectural style) and function (what the program is meant to do). Perhaps Object Oriented architects tend to focus on UML Class Diagrams more than on UML Activity Diagrams?

Unlike Object-Oriented design, SOA is tailored to solve problems in a specific domain. As Miko Matsumura puts it:

"there are nearly an infinite number of software programming problems—game programming, graphics, engineering, mathematics, statistical modeling, you name it. The SOA that is getting people all hot and bothered isn't particularly interesting as a solution for all of these other problems. SOA is for business."

If this begins to sound a lot more like a Business Architect than a Technical Architect... I've made my point.

Continue Reading...

Can I call you back? - Asynchronous Web Services

"Many of the benefits of web services can't be realized until asynchronous interaction becomes well understood and widely practiced, and some high-value applications can't be deployed properly or at all without it."

A more elegant approach for implementing an asynchronous web service is to pursue a callback approach; the client makes a request and leaves a return address (where the service should send the response). The obvious drawback to the callback approach is the limitation that it places on the client; the client must have a mechanism that allows it to receive messages.

Best practices suggest that you should support both polling and callbacks when implementing an asynchronous web service. Supporting both approaches will make your service accessible to the widest audience of clients, and that's a good thing.

Web Service Addressing defines standard SOAP headers for request and response messages. Most web service containers do not directly support this specification yet, but hopefully that's changing.

What would really help the average Java business programmer would be a "standard" technique for decoupling the code for implementing each asynchronous technique from the code that actually implements the business functionality of the web service. Why clutter up business logic with what is essentially messaging logic?

JSR 181 defines metadata for injecting web service aspects into POJOs, but it does not appear (to me at least) to provide for automating the creation of dual implementations (polling and callback).

My general feeling is that we are on the cusp of ubiquitous asynchronous web services, but we aren't quite there yet. It's still mostly a "roll-your-own" frontier, but the tracks have been laid and civilization is sure to follow.

Continue Reading...

Service Orchestration vs. Service Choreography

Do you know the difference between Web Service Orchestration and Web Service Choreography?

The distinction between WS-Orchestration and WS-Choreography is important to understand, but unfortunately the vocabulary that we are defining for dealing with web services and SOA is... uh... (How shall I put it?)... unhelpful.

One definition that I found for "orchestration" on the web is the following:

"Orchestration or arrangement is the study and practice of arranging music for an orchestra or musical ensemble. In practical terms it consists of deciding which instruments should play which notes in a piece of music."

A similar search for the definition of "choreography" returned the following:

Choreography is: "the arrangement and movement of performers onstage; though the term cutomarily applies to dancers, it is also used to denote the orchestrated movement of actors, especially in stage combat"

Based on these definitions, Orchestration is about music and Choreograhy is about dance... but for some odd reason this doesn't help me grok the distinction between WS-Orchestration and WS-Choreography.

The real distinction lies not in the dictionary, but in the differences between the Business Process Execution Language (WS-BPEL) and the Web Services Choreography Description Language (WS-CDL).

Orchestration == Executeable Process

Web Service Orchestration relates to the execution of specific business processes. WS-BPEL is a language for defining processes that can be executed on an orchestration engine.

Choreography == Multi-party Collaboration

Web Service Choreography relates to describing externally observable interactions between web services. WS-CDL is a language for describing multi-party contracts and is somewhat like an extension of WSDL: WSDL describes web services interfaces, WS-CDL describes collaborations between web services.

Ah, if only the "C" in WS-CDL was for "Collaboration" instead of for "Choreography"... life may have been a little simpler ;-)

SOA/ESB Level Set

	This figure shows the basis of an application developed using SOA principles and an ESB. Specific services are plugged into the bus, making them available to any application.
The service bus is responsible for routing messages to and from each service, eliminating the problems associated with "point-to-point" programming. Many ESBs are built on top of reliable messaging technologies. Messages are often XML based, but some ESBs allow for multiple protocols and formats. Applications are composed by developing driver programs that orchestrate the available services. Languages such as BPEL are tailored for writing these "driver" programs. Any number of "driver" programs can be written to utilize the services on the bus.
	If the application requires a user interface it is generally bolted on to the driver program.
Any user interface technology can be used; Rich Interfaces using Swing, Browser Interfaces using JSF and/or AJAX.
	If the application requires an external interface to the world beyond the service bus, those interfaces are also generally bolted on to the driver program.
Although it is certainly possible to expose all of the services to the outside world, that is usually not desirable. Generally only "composite services" that represent a service offered to a business partner will be exposed. Before exposing any service to the outside world, security issues must be addressed.
	Legacy programs are added to the mix by creating custom “service adaptors” between the legacy code and the service bus.
The success of integrating any legacy asset into the mix is related to the difficulty of crafting an adaptor (a.k.a. connector) for that asset. If the adaptor is fairly straight-forward, successful integration is more likely. If the adaptor is convoluted, only minimal success is likely. Although reuse of legacy assets is touted as a major SOA selling point, in reality many legacy assets will require refactoring to be of much use. The real payoff is adopting a style of programming that will make new services much easier to reuse later.
	The services that are plugged into the bus should be loosely coupled. Invoking one service should ideally have no side effect on other services. Care must be taken if the services directly access a shared database.
If services share records in a common database, they are coupled in a manor that may not be obvious to the author of the driver program. "Hidden" coupling in an underlying database can be a real problem. When possible, services should be stateless and operate only on the documents that are passed to them.
	To avoid hidden database coupling, some vendors recommend plugging a data service into the service bus.
The primary concern with the data access service approach is the performance issue associated with accessing all data through the bus. The use of data access services has let to several related ideas, such as using Service Data Objects to pass working data between services.

Please let me know what else you think I should add...

Big dreams on the longest night of the year...

The past year has seen a lot of progress in enterprise computing. At the beginning of this year I was writing an "elevator speech" on Service Oriented Architecture to introduce the concepts to my IT colleagues:

"This talk will be far more evangelical than technical, and I hope that it will de-mystify SOA for some. I'm sure many of you will say "Duh!" when you read some of the points, but you'd be surprised how many folks just don't get it (yet). "

The other big story of 2005 has to be AJAX (Adaptive Javascript and XML). I first blogged about AJAX in March after witnessing an amazing demo by Dion and Ben. I was wowed by the possibilities, but haunted by:

"memories of pre-custom tag JSP pages... a little puddle of HTML markup embedded in an ocean of Java code (only this time it's JavaScript)"

AJAX and Service Oriented Architectures are fond remembrances of the recent past... What about dreams for the future?

David Gelernter said the following during an interview for the Sun Developer Network:

"Huge numbers of "non-technical" people rely on computers not because they want to but because they have to. The great author and culture-critic George Orwell noted 60-odd years ago (and I noted in Mirror Worlds) that some people like playing with machines; some people don't. People who don't are just barely hanging on today. They need something far simpler, far more powerful than they've got today"

I think that SOA and AJAX are catalysts for the "far simpler future" that most people need. At first glance, you may not see the linkage, but take another look.

Web Services are what makes AJAX sexy. AJAX is what makes Web Services accessible to the masses.

As the SOA mindset takes hold, we should see an explosion of web services. As the AJAX toolsets mature, it should become incredibly simple for "power users" to craft tailored solutions from the new bounty of web services. Throw in improvements in service orchestration and choreography (BPEL for People anyone?), and I think we'll see a new generation of Renaissance Artist/Programmers.

In a recent interview, Bill Buxton offered the following scenario:

"The interesting thing is if you can throw a cell phone on a passenger seat, the phone rings, and the phone says to the stereo, "There's a call coming in. Can you turn the music down and can I borrow the speakers? And by the way, can I use the microphone embedded in the steering column?" Now you can just talk hands-free."

In Bill's example, the stereo provides an "audio output" service, the the microphone provide a "voice input" service, and the phone provides several communication and choreography services. The key is to help the "Renaissance power user" discover these services and then to help wire the services together on-the-fly. The pieces of the puzzle exist, the challenge is to make it easier to put them together.

The languages, tools, and devices that deliver this kind of power to the new Renaissance men and women are what I plan to dream of tonight...

Pleasant dreams indeed ;-)

Form Validation in an SOA context

Validation of user input has always been an important aspect of user interface programming. You have to make sure that the values a user provides to your program are of the right type, in the right range, etc. and you should provide meaningful feedback to help users correct any errors.

I have often blogged my opinion that validation rules are business logic, and that they should not reside in the presentation layer. The presentation layer may apply validation rules, but developers should not have to modify presentation layer code (including JSPs) when validation rules change.

Please note that when I use the terms “validation rules” or “validation logic”, I am incorporate all of the following:

• Field level validation: rules applied to a field based solely on the contents of that field
• Form level validation: rules applied to fields of a form based on the contents of other fields on the form
• Application level validation: rules applied to fields of a form based on data that is external to the form

Assume that you craft a business service called ProcessLoanApplication. This is a coarse-grained service that provides a few service methods to achieve a specific business goal, and the service involves the exchange of large document-oriented parameters.

ProcessLoanApplication is designed for B2B integration with an external partner. Our assumption is that a “LoanApplication” document is generated by the partner’s systems, and transmitted to ProcessLoanApplication when complete. Obviously, we’re going to have to apply our validation rules to the incoming documents, and we will have to provide document-oriented feedback if any rules are violated.

So far so good, but now assume that that our own company wants to provide a browser-based application that allows users to create “LoanApplications” and submit them for processing (via the ProcessLoanApplication service).

Using an IDE such as Java Studio Creator, it is fairly straight-forward to create a browser-based application to front-end a web service. We can easily configure JSF controls to gather the necessary input data, prepare an appropriate document, and invoke the ProcessLoanApplication service. If ProcessLoanApplication detects any errors, they will be returned in a document to the JSF application. The response can then be parsed and we can display appropriate error messages to the user.

If you are not already shaking your head at this description, then you probably have not done much user-interface development. It is generally considered as “bad form” to defer all input validation until after a form is submitted. The consensus view is that you should detect invalid input data as soon as possible and prompt the user to correct the mistake.

Assuming that we want to provide timely feedback to the user, and that we don’t want to duplicate input validation logic in our JSF application, we will need a service interface to apply (or obtain a copy of) the validation rules that are used by the ProcessLoanApplication service.

The need to obtain a copy of our validation rules becomes more apparent if we also encounter an external partner that wants to apply our rules to their documents prior to invoking the ProcessLoanApplication service. Within our own enterprise, invoking a validation service as the user inputs data works fairly well (sort of an AJAX approach), but constraining 3rd parties to operate in that manner may not be acceptable (for example, they may want to minimizes transmissions through their own firewall).

This is one of those cases where I really do not want to roll-our-own solution. I want proven tools and templates to insure that the business oriented tasks (writing business logic, writing validation rules, defining workflow, etc.) do not get intertwined with infrastructure development and configuration. I know that all of the pieces of the puzzle exist, but I am still looking for a holistic solution that addresses the client, the web application, and the web service.

Are there widely-used blueprints, best-practices, frameworks or products that address the scenarios that I have outlined in this blog? Are there de-facto standards for creating validation services that are intended for use both by coarse-grained web services, and by JSF-centric web applications? Are there de-facto standards for exchanging validation rules with external partners (particularly application level validation rules)?

If you have any suggestions (besides changing careers) I would love to hear them.

Decoupling client-side JavaScript components from server-side web frameworks

I had dabbled with JavaScript years ago, but the pain of dealing with IE, and particularly Netscape 4, left a bad taste in my mouth. I settled for pretty dull web interfaces, using only minimal DHTML and avoiding "fancy client-side stuff" like the plague. Most of my presentation layer logic was server-based, using frameworks like Struts to handle dynamic content and validation logic.

Web component frameworks like Echo, Tapestry and JSF added spice back to life. Wonderful people took all that nasty JavaScript and packaged it up in nice little bundles that I never had to deal with. I could now add snazzy menues, tree controls, and sortable tables to my web-apps without having to stray far from my comfortable and safe Java knowledge base.

Life was definitely better, but I had a nagging suspicion that I was living a lie. JavaScript is the native tongue of browsers, and I was crippling myself by speaking only Java and XML. This suspicion was verified when I started trying to port components between web component frameworks. Let's face it, the "sexy" behavior of any Tapestry or JSF component is written in JavaScript... the server-side Java that processes the requests is pretty mundane. If you can't reuse the Java, no big loss. If you can't reuse the JavaScript, you're starting over from scratch.

My journeys between Tapestry and JSF may have been enough to rekindle interest in JavaScript, but what really got me hooked again was AJAX. Many blogger have pointed out that AJAX is nothing new, but something surely changed. Sexy web-apps that once required applets or Flash are popping up all over.

Take a look at the demo for Novell's Hula project. That's a truly sexy user interface, and it's implemented as JavaScript over Web Services. This is a pure client/server (or should I say client/services) architecture: JavaScript client and Mono services.

The folks who are writing these sexy apps know JavaScript/DHTML and leverage it to the max, but what really interests me about this architecture is the decoupling of the JavaScript from the server-side code. The Hula client runs on Web Services that happen to be implemented using Mono, but you could change the implementation to Java and the client wouldn't care.

Contrast the clean separation of Client and Service from approaches like JSF and Tapestry. With Tapestry/JSF, an HTML author designs the layout, and a Java author implements the user interface logic. With the Client/Service approach, a DHTML/Javascript author writes the user interface, and a Java/Mono author writes the services.

With the JSF/Tapestry approach, there's a tremedous temptation to tightly couple the services to the user interface... ending up with a great application, but not with reusable services.

With the Javascript Client/Java (or Mono, or PHP, etc) Service approach, it's much more likely that your services team will develop reusable services, and you'll probably end up with several tailored user interfaces.

I don't think that Tapestry or JSF are "doomed", but I do think we're going to see much more interest in DHTML centric client frameworks. The clever person who develops a server-side framework that appeals to JavaScript programmers is probably going to be very popular.

The bad news is that the IDEs for JavaScript just aren't very good yet, but there is a lot of activity going on. With better tools, better Javascript SOAP support and JavaScript embedded in Java 6, we might even start seeing more server-side JavaScript servicing those rich JavaScript clients... a pure JavaScript web application stack.

I'm not about to abandon Java, but I am going to learn more JavaScript... When in Rome, it really does help to speak Italian.

Uncertainty may not be what you think it is

Tom Koulopoulos was the keynote speaker at a conference that I attended last week. He's a really bright guy and a very entertaining speaker (or at least I think so).

Tom covered a wide range of topics, one of which was in dealing with uncertainty. He began with a question to the audience:

"What are some games that involve uncertainty?"

Audience members tossed out three examples:

1. Dice
2. Cards
3. Chess

Sure enough, Tom's next slide showed the same three games (He swore he didn't plant shills in the audience, but you never know).

We fell right into his trap. None of these games really involves uncertainty. For example, with dice you will never roll a 13 (Assuming that you are using two "standard" dice).

Uncertainty is not what you least expect, it's what you don't expect at all.

So how do you plan for uncertainty? I can tell you from my own experience that the answer is not "plan for every eventuality". In the first place, you can't know "every eventuality". In the second place, if you try to add in hooks to deal with anything that might happen your product will be bloated and over-engineered. In the third place, the hooks that you build will seldom be the ones you need.

Agile methods evolved from the recognition that you can't predict the future:

• Build only what you need today
• Build it lean
• Build it well
• Get really good at building things fast

If you can build things quickly, you can respond quickly when the thing that you never expected happens.

I have great respect for Bruce Tate and his Better, Faster, Lighter mantra, but in my world of application integration I have to focus at a different level. The Spring/Hibernate stack is great for implementing point solutions, but it's not enough for implementing connect the dots scenarios. Put another way, Spring/Hibernate is great for creating individual services, but it's not the answer for implementing solutions that require orchestrating dozens of services.

Einstein is quoted as saying:

"Make things as simple as possible, but no simpler"

I think that Spring/Hibernate solutions by themselves are too simple for most enterprise-wide problems.

Spring/Hibernate articles generally discuss solutions that are created from scratch. Enterprise-wide solutions evolve over time, and they're never pure. Just as our DNA contains garbage sequences, enterprise-wide solutions always contain legacy junk that will never quite die.

There is no "homogenous future". There will never be a single language, operating system, database management system, or presentation technology.

We need a better, faster, lighter approach towards integrating heterogeneous services.

This is where SOA comes in. SOA mandates loose coupling between services. Loose coupling leads to less efficient solutions, but the agility of the solutions to respond to changing requirements is preserved.

If you tailor a suit too well, it will only fit one man. The suit will look great, but as the man gets older (and middle-age spread sets in) the suit will probably end up at a thrift store.

SOA is a paradigm that allows for growth. OpenEAI is a Java-based open-source framework for implementing solutions that can grow.

Stephen Wheat and Tod Jackson presented a session on OpenEAI at the conference that I attended last week. The framework and methodology were developed by the University of Illinois and then released publicly under GNU licenses.

From their web site:

In the face of this complexity, it is very helpful for an enterprise to have an integration methodology that can be uniformly applied to any new integration project, regardless of the technology involved. Even more helpful is a flexible technology foundation that has already been applied to many types of projects and can be readily extended and applied to new situations. The OpenEAI Project strives to provide this methodology and technology and, most importantly, to provide it in a way that allows all the individuals and enterprises that practice the methodology and use the technology to share their experience and benefit from each other's work. The goal of the OpenEAI project is to explore and codify the science of enterprise application integration in a practical, public, and methodical way.

I don't know if OpenEAI will be to SOA what Spring/Hibernate is to Agile methods, but it looks like a good start.

Uncertainty may not be what you think it is

Tom Koulopoulos was the keynote speaker at a conference that I attended last week. He's a really bright guy and a very entertaining speaker (or at least I think so).

Tom covered a wide range of topics, one of which was in dealing with uncertainty. He began with a question to the audience:

"What are some games that involve uncertainty?"

Audience members tossed out three examples:

1. Dice
2. Cards
3. Chess

Sure enough, Tom's next slide showed the same three games (He swore he didn't plant shills in the audience, but you never know).

We fell right into his trap. None of these games really involves uncertainty. For example, with dice you will never roll a 13 (Assuming that you are using two "standard" dice).

Uncertainty is not what you least expect, it's what you don't expect at all.

So how do you plan for uncertainty? I can tell you from my own experience that the answer is not "plan for every eventuality". In the first place, you can't know "every eventuality". In the second place, if you try to add in hooks to deal with anything that might happen your product will be bloated and over-engineered. In the third place, the hooks that you build will seldom be the ones you need.

Agile methods evolved from the recognition that you can't predict the future:

• Build only what you need today
• Build it lean
• Build it well
• Get really good at building things fast

If you can build things quickly, you can respond quickly when the thing that you never expected happens.

I have great respect for Bruce Tate and his Better, Faster, Lighter mantra, but in my world of application integration I have to focus at a different level. The Spring/Hibernate stack is great for implementing point solutions, but it's not enough for implementing connect the dots scenarios. Put another way, Spring/Hibernate is great for creating individual services, but it's not the answer for implementing solutions that require orchestrating dozens of services.

Einstein is quoted as saying:

"Make things as simple as possible, but no simpler"

I think that Spring/Hibernate solutions by themselves are too simple for most enterprise-wide problems.

Spring/Hibernate articles generally discuss solutions that are created from scratch. Enterprise-wide solutions evolve over time, and they're never pure. Just as our DNA contains garbage sequences, enterprise-wide solutions always contain legacy junk that will never quite die.

There is no "homogenous future". There will never be a single language, operating system, database management system, or presentation technology.

We need a better, faster, lighter approach towards integrating heterogeneous services.

This is where SOA comes in. SOA mandates loose coupling between services. Loose coupling leads to less efficient solutions, but the agility of the solutions to respond to changing requirements is preserved.

If you tailor a suit too well, it will only fit one man. The suit will look great, but as the man gets older (and middle-age spread sets in) the suit will probably end up at a thrift store.

SOA is a paradigm that allows for growth. OpenEAI is a Java-based open-source framework for implementing solutions that can grow.

Stephen Wheat and Tod Jackson presented a session on OpenEAI at the conference that I attended last week. The framework and methodology were developed by the University of Illinois and then released publicly under GNU licenses.

From their web site:

In the face of this complexity, it is very helpful for an enterprise to have an integration methodology that can be uniformly applied to any new integration project, regardless of the technology involved. Even more helpful is a flexible technology foundation that has already been applied to many types of projects and can be readily extended and applied to new situations. The OpenEAI Project strives to provide this methodology and technology and, most importantly, to provide it in a way that allows all the individuals and enterprises that practice the methodology and use the technology to share their experience and benefit from each other's work. The goal of the OpenEAI project is to explore and codify the science of enterprise application integration in a practical, public, and methodical way.

I don't know if OpenEAI will be to SOA what Spring/Hibernate is to Agile methods, but it looks like a good start.

The Same Big Things

Computing has come a long way since the 1950s, but the constituencies acknowledged by those "2nd Generation" languages are still important to remember: The needs of people who use computers to accomplish a task are different then the needs of people who build the infrastructure.

The 3rd generation languages like Java are far richer then their predecessors, leading many of us to believe that we don't need distinct languages for users and builders any more. I suspect that belief may have caused confusion for the "builders" and unintended complexity for the "users".

SOA is a good case in point. I've heard "builders" say that SOA stands for "Same Old Architecture", meaning that there's nothing new here; SOA is just a new buzzword for marketing types. From the "builders" perspective, this is partially kind-of sort-of true, but from the "users" perspective nothing could be further from the truth.

SOA is a architectural style that is tailored for executing Business Processes. I'm sure that the "Service" focus of the acronym is due the the suitability of Web Services as building blocks, but the whole point is to make it easier to automate Inter and Intra Enterprise Business Processes (BPOA would be a much more descriptive acronym).

"Users" (in this case corporate developers) need much different tools for taking advantage of SOA then the "Builders" who are developing the infrastructure for SOA. SOA "users" need tools geared towards defining and debugging Business Processes... and this entails much more then simply defining and processing Business Forms.

One of the drawbacks with most existing J2EE applications is the inability to extract the Business Process and Business Rules from the running application. The Business aspects are lost in the sea of details that make up the run-time environment. This is a huge problem for businesses; a simple change to a business process can be extremely expensive to implement.

I think that one "Big Thing" that we need is to put the "P" in COBOL (figuratively speaking): We need a COmmon Business Process Oriented Language. We need to make a clean break between the code that implements the Business Processes and Business Rules and the underlying infrastructure. The BP infrastructure will still be written in languages like Java, but the Processes and Rules will be written in Business Oriented Languages by corporate developers.

"What would happen if I made this change to my process?"

The infrastructure that will "run" Business Processes won't be implemented in a single language on a single platform. Business Processes span multiple Business Partners; heterogenous platforms are going to be a fact of life. Many of the building blocks for this infrastructure are already in place, but the whole enchilada is far from finished.

Gregor Hohpe's ramblings are as good a place as any to see just how far we are from finishing the infrastructure that we will need. For example, today's BPM tools (like those from Oracle and BEA) almost all require a central "controller" for the Business Process, but this just doesn't match reality. Business Processes are a lot like conversations, you make an inquiry and expect a response... but that might spin off a whole new thread that you now nothing about. Building an infrastructure that can support complex processes is a daunting task.

Searching for the perfect Business Process Language is sort of like the searching for the Holy Grail. BPEL is a good start, but it still doesn't handle some common scenarios.... which is why BPMI.org and others are already working on BPXL. The Workflow Management Coalition's Extensible Process Description Language (XPDL) or UML Activity Diagrams might be a better starting point... but the jury is still out.

Bruce Silver's article "Agile To The Bone" is a very good summary of what's up in standards land.

Continue Reading...

Objects, Components, Web Services and BPEL

Robert Sessions has published a good article discussing the “fuzzy boundaries” between object, components, and web services. The following table from his article pretty much sums up the key differences with respect to performance:

This table can be useful in determining whether or not you should provide a Web Services interface for a particulatr Object or Component, and as Robert points out, myriad tools are available to generate the necessary wrappers. All you need to do is to determine whether or not the wrapper is appropriate.

Pondering these differences once again prompts me to beat on a dead horse: Something's just not right about BPELJ.

Standard BPEL allows you to orchestrate distinct Web Services into Business Processes. The Web Services protocols are the only means by which standard BPEL engines can interact with Services. If one of your Services is written in Java, you will have to wrap it in a Web Service to add it to your Buisness Process, and you will suffer the performance penalty associated with Web Services.

BPELJ was developed to give developers the ability to directly invoke Java Services from within BPEL “scripts”. No need to wrap your Service, and no performance penalty of marshalling and unmarchalling XML. Nice idea, and if you are using tools such as those from BEA, IBM, and Oracle you won't even have to deal with ugly syntax.

My beef, legitimate or not, is that BPELJ a myopic solution. BPELJ works because the developer has prior knowledge that a specific Service has a Java interface. If an instance of the Service is ever created without a Java interface, then that version won't be eligible to be incorporated into the Business Process (unless the process is edited).

Here's a scenario; You write a BPELJ process for your company using a Service from ACME corporation that is written in Java. Another company, EMCA, comes on the scene, but they implement the Service as a Web Service. You might consider this scenario as job security since you get to go back and update your BPELJ script, or you may consider this scenario a pain. I hold the latter view.

Forget BPELJ; We should make BPEL engines smarter. BPEL engines should optimize performance based on “learning about” the interfaces that are available for specific Services. To make this work, each Web Service would have to indicate alternate interfaces that it supports (Java RMI, C#, Jini, etc). In my example, the ACME Service provides both a Java and a Web Services interface, while the EMCA Service only provides a Web Services interface. When “talking” to the ACME Service, the “smart” BPEL engine can use Java protocols to speed things up.

BPEL is about Business Processes. From the perspective of the Business Process Author, the implementation of the Services is immaterial. No company is going to change a Business Process because one partner likes Java and another likes .Net. Implementation details, such as the interface that is used to interact with a Service, just don't belong at this level.

Programmer's have been very successful in automating optomization in the past. Witness the JIT compilers and the optimization logic in modern processors. Why should BPEL engines be any different?

The SOA Elevator Speech

Here are some notes from a "brown bag" talk that I am preparing for our IT staff, many of whom are died-in-the-wool mainframe COBOL programmers. This talk will be far more evangelical then technical, and I hope that it will de-mystify SOA for some. I'm sure many of you will say "Duh!" when you read some of the points, but you'd be surprised how many folks just don't get it (yet).

I like the following definition of SOA from a paper by Bernhard Borges, Kerrie Holley and Ali Arsanjani, but it's a bit over-the-top as an introduction:

SOA is the architectural style that supports loosely coupled services to enable business flexibility in an interoperable, technology-agnostic manner. SOA consists of a composite set of business-aligned services that support a flexible and dynamically re-configurable end-to-end business processes realization using interface-based service descriptions.

I think it works better if I break down the definition as follows:

SOA is an architectural style that encourages the creation of loosely coupled business services

Loosely coupled services that are interoperable and technology-agnostic enable business flexibility

An SOA solution consists of a composite set of business services that realize an end-to-end business process

Each service provides an interface-based service description to support flexible and dynamically re-configurable processes

I’d like to get across the following points about SOA:

1. SOA isn’t really new, but there are now some standard technologies (such as Web Services) that make it much easier to implement
2. The “Services” in SOA are business services… updating a loan application is a business service, updating a record in a database isn’t
3. Services are linked together to implement business processes... Business Process Engines make it easier to combine services into business processes, and BPEL is an emerging standard language for this purpose
4. Business partners can use your company's services within their own business processes and your company can use services provided by business partners within your own business processes.
5. SOA solutions favor flexibility over efficiency... machine cycles and network traffic are less important then being able to quickly implement and change business processes

On the implementation front, we need to clear up the following common misconceptions:

1. Services aren’t tied to user interfaces. User interfaces invoke services. Many user interfaces can use the same service. A partner may build a user interface on top of a your service, and you may build a user interface on top of a partner’s service.
2. Services can be implemented in any language, COBOL, Java, etc., but all services must support the same invocation/communication protocols (for example XML/SOAP).

Perhaps that's a bit more information then you can get across in one elevator ride, but it's close.

If you could add one more point, what would it be?

This work is licensed under a Creative Commons License.