John Reynolds's Blog

Community: NetBeans Archives

Net Beans (6.1) Page Flows

"We don't see the need for BPM"

Several folks said "We don't see the need for BPM tools"... and I would like to address that in this blog entry.

I'm really not surprised that the average Java developer doesn't see a need for BPM... In fact I'm not surprised that the average worker in any industry doesn't see the need for BPM.

All of us who work for a living participate in business processes. All of us who interact with businesses participate in business processes.

Almost of us participate in business processes. We don't think about it much because we generally don't care what the full process is... We generally only care about the tasks that we have to perform within any process.

For example: When you apply for a loan you are a participant in a "Loan Origination Process". In general you really don't care about all of the steps of the process. You don't care about all of the tasks that the lender has to perform. You only care about the process step where you fill out the loan application and the process step where you respond to the lender's decision.

When you develop software you are a participant in a "Software Development Process". Speaking for myself, when I am working in "programmer mode" I usually don't care about all of the steps of the process. I only care about understanding the requirements, writing the code, and fixing any bugs that QA finds.

My Project Manger (PM) cares about many more steps of the software development process than I do. Project Managers have to coordinate the programmer tasks and those of everyone else on the project. Most of the PMs that I've met don't think of their projects as instances of the software development process... They tend to view their projects as schedules, which explains why those lovely Gant charts in Microsoft Project are pretty much all that they need in terms of tool support.

It's only when you get to a much higher level in your organization, perhaps to the VP of Development, that you find someone who might really worry about the whole Software Development Process. If your organization is large enough, the VP might be responsible for several projects that are running at the same time, with each project at a different point in the process.

As long as all of your company's projects are completed on time and under budget, your VP is probably going to leave your Software Development Process (SDP) alone. For some strange reason, all of the VPs that I have worked for kept tweaking their SDPs.

Development VPs mess with their SDPs to try to lower costs, to have more predictable schedules, and to just generally drive their employees crazy. They want to make more money. Meddlesome Development VPs like to find problems in the process, "improve" the process, and then do it all over again.

Your company has lots and lots of business processes besides your Software Development Process. Some are really mundane, some our critical to your company's bottom line.

All of your Corporate Officers are really interested in managing all of those business processes. Inefficient processes cost more, both in terms of expense and in terms of lost revenue, than efficient processes.

Managing a process...

The first step in managing a process is to know what the process actually is. This includes the steps that are done by people and the steps that are done by machines. I once found it necessary to gather a dozen people in a room to find out what everyone thought the process was, and then later discovered the process wasn't at all what they thought it was. Big companies like to spend lots of money to diagram their processes... which they then file away for five years until they do it again.

The second step in managing a process is to gather metrics... How long (on average) does it take for each step of the process to take? How often are steps repeated (presumably due to errors)?

The third step and fourth steps in managing a process are to analyze and actively manage the process. Use the metrics that you've gathered to analyze process performance, develop a better process, try it out and start all over again. That's Continuous Process Improvement.

All good BPM suites provide infrastructure support for all the steps needed to manage processes...

First: Know the process

At the heart of all BPM suites is a process engine that executes process definitions. These definitions are usually represented graphically using BPMN notation. Process diagrams are usually generated in conjunction with the business people who own the process... Please note that it's much more important that the business folks understand how to "read" the process diagram than it is for them to be able to draw the diagram... The process engine is going to execute the diagram, so if it's wrong the process will be wrong.

I like to say that processes are composed of Human Powered Services and Autonomous Services... Most BPM suites provide tools that help you build the Human Powered Services. These tools are basically form designers, and they're good for doing many business-task-oriented user interfaces, but they aren't meant to be used for designing "filthy rich" client interfaces... All of the good BPM suites provide interfaces that allow you to hook in virtually any client that you wish to construct. If you want a Swing application to perform a task, then you can write a Swing application and hook it in. If you want to use Netbeans or Eclipse to perform a task, you can do that too.

BPM is not about replacing all of the applications that you use. It's about coordinating the use of those applications to perform a process.

But back to knowing what the process is.... You know what the process is because you can always see it. The process definition is not scattered across multiple applications and background tasks. It's contained in one precise definition.

Second: Gather metrics

Since the process engine is managing the process, the process engine knows when each task is started and ended. The process engine knows who performs each task. This makes it incredibly easy to instrument a process.

Third: Analyze the process

All good BPM suites provide tools for analyzing all of those nifty metrics that you've gathered. The really good suites even let you use your historical data to simulate how a proposed process change would have performed in the past.

Fourth: Modify the process

With that analysis in hand, you can now go back and modify your process... This is really easy because, as you may remember, the process definition is in a single place.

I cannot emphasize how useful it is to have that process definition driving everything else. In essence, you have a process-centric view of your code base. In my pre-BPM days I would join a project and have to spend weeks to track down the code that was responsible for some specific process behavior. With a BPM tool, you can always start at the process diagram and "drill down" to the code you need to change.

But can't I just put all these features together myself?

Of course you could. What do you think all the BPM vendors did?

Wrong question... Will a client pay you to put all of these features together AND pay you to develop a process on top of that?

The optimist in me says: "Why of course they will!"

The realist in me says: "Why would they?"

Sounds boring... Why shouldn't I just ignore BPM?

There's no reason to pay any attention to BPM as long as you don't make your living building process-centric applications.

If the applications that you write only involve one participant, you'll never need BPM... but I encourage you take a good hard look at those applications because they may actually function better if they are process-aware.

For example... Every issue-tracking system implements multi-participant processes. Sadly the processes are usually hard-coded in these tools, but more and more you'll find that issue tracking tools (like Jira ) are providing web-service interfaces to interact better with the other tools that are used in a wider process.

Another example... Your favorite IDE. More and more we are seeing process oriented functionality creeping into these tools. Why not leverage an existing process engine (like Apache ODE ) and make those processes more flexible?

Parting thought...

From the very beginning, the core of most programming has been Process Definitions and Data Structures. BPM is our latest reminder of that.

Why do Java developers hate BPM?

Java developers hate BPM.

The preceding sentence is (of course) intentionally tailored to be controversial. People tend to read controversial blogs, and I'd like you to read this one. Now that I have hopefully grabbed your attention I'll tone it down a bit...

A lot of Java developers hate having to use BPM tools instead of the object oriented tools that they are comfortable with.
I work on a lot of BPM projects, and I work with a lot of other folks who work on a lot of BPM projects, and we have all encountered resistance from traditional Java developers. Java developers (in general) would rather use frameworks like Struts and Spring than be saddled with the constraints of a BPM suite.

Java frameworks like Struts and Spring are in the background... they provide just enough support to "set your creativity free" so that you can be a real programmer. You can build almost anything with Spring or Struts (if you've already mastered the intricacies of Java). They are light-weight, they're agile, and they look sexy on your resume.

BPM suites are in-your-face. They rob you of your creativity. They dictate to you how you will develop your application.

BPM suites make programming boring. They force you to use point-and-click and drag-and-drop tools to design your process diagrams, data models and forms. What's worse, they actually encourage Business People to model processes and design forms on their own... Fortunately most Business People are too intimidated to use these tools, but it does open the door for them to look over our shoulders and meddle in our affairs.

That certainly doesn't sound like something that real programmers would like, does it?

I'm not being subtle, am I?

BPM suites are a threat to traditional Java programmers. These suites are far from perfect, but even in their current state we can see where things are heading. The days of the Java Guru as indispensable are fading... We've used Java to build tools that make knowing Java itself less important, and that's opened up competition for us from folks who didn't spend years learning Java.

We're victims of our own success... and that's going to cost us.

That's why Java developers hate BPM.

Human-Powered Web Services

As SOA has become a more mainstream IT concept... you might even say a more mundane IT concept... there has been a lot of thought put forth on the creation of Composite Applications.... collections of distinct Services that are orchestrated and choreographed together (using BPM technologies) to implement some "higher-level" business functionality.

Many of these Composite Applications need user-interfaces, just like "normal" applications, but fortunately the transition from building a UI on top of objects (often EJBs) to building a UI on top of Services (often Web Services) is generally not that hard for programmers to make. Perhaps the biggest adjustment is in moving from a world where pass-by-reference was possible to a message-oriented world where everything is pass-by-value... but that's not really much of an obstacle for good programmers.

A more subtle aspect of user-interfaces and Composite Applications becomes apparent when you start to consider the Services that make up the applications. Most of the folks that I talk with tend to think of Services as automatons... fully automated tasks carried out by machines. This is especially true when I use the term Web Services... Everybody knows that a Web Service has nothing to do with people, it's just a machine at the "other" end of the line.

I'm going to suggest that conceptually limiting Web Services to automatons is a mistake that has hampered our ability to benefit from all that SOA has to offer. Our shared belief that Web Services are "always" automatons has impacted our conception of Composite Applications and has even impacted the design of most of our development environments (IDEs).

That's a pretty strong statement, so I'd better try to explain myself. Perhaps an example is in order....

Let's say that you are building a Composite Application to process an Order of some sort. Also assume that your customers speak many languages, and that it's necessary to translate each order to a common language before it is processed. To make this process pretty darn simple, let's also assume that Services are available to translate the orders from one language to another. Piece of cake... just funnel the order through the Language Translation Service and you're done.

Any good Service-Oriented IDE (such as NetBeans with the Enterprise Pack ) can handle the above process with ease. The IDE's tools make it relatively simple to orchestrate the process using BPEL, and relatively simple to build a UI on top of the BPEL (it's a Web Service after all). In short-order, you've created your Composite Application and all is well in the world.

So where's the beef? What's wrong with this picture?

Nothing really... until you start to consider the creation of the Language Translation Service with your IDE. Our conception of Web Services as automatons causes few problems when consuming the Services, the problem becomes evident when creating them.

I picked Language Translation on purpose... It's hard. Go out to any of the free translation services, translate this page to another language and back again, and you will see what I mean. Language Translation (at this point in time) is better performed by a person than by a machine.

Here's the previous paragraph translated to Spanish and back to English:

Try building a Human-Powered Web Service with your IDE. You can do it, but it's tough. You're going to have to cobble together something that links together a human-oriented application with a Web Service, and it's probably not going to be pretty. This "ugliness" really shouldn't be a big surprise, because we've never thought of doing such a thing (maybe "never" is a bit strong... but not very often).

Now that I am spending much more time as a Business Process Guy, I'm finding the need for Human-Powered (Web) Services isn't rare at all. All of the good BPM suites (like Lombardi and Fuego) erase most of the distinctions between creating Human-Powered Services and creating Autonomous Services because most business processes are a mix of both types of Services.

Unfortunately, at this time there aren't widely-adopted standards for Human-Powered Services (HPS)... the BPM suites are proprietary and the concept of a "stand-alone" HPS seems alien. There's a good reason for that... when an HPS is invoked, the Human has to "know" that there is work to be done and that implies a notification mechanism like a task list or email.

I don't want to imply that it will be "easy" to come up with standards for Human-Powered Services... just look at all of the turmoil surrounding "BPEL for People"... but it is time for us to start addressing this class of Service in our IDEs. In this blog entry I have focused mostly on the creation of the HPS... but there are also some issues of consumption (people tend to be much slower than machines).

What it all really boils down to is wider awareness of Process-Oriented concerns... business processes are made up of automated and human-performed tasks, and we need to be able to implement both types of tasks in a manner that allows them to be consumed by a multitude of applications. That's what SOA was really all about in the first place.

JSF and AJAX versus Swing

Evan Summers wrote a very good blog on "Swing versus everything else" a few days ago, and it started me thinking...

Many heated battles have been fought in the war between browser-based applications and "stand-alone" applications... and when a new skirmish flares up it often brings to mind lyrics written by (Country) Joe MacDonald back in the 60's:

"And it's one, two, three,
What are we fighting for?
Don't ask me, I don't give a damn,
Next stop is Vietnam;
And it's five, six, seven,
Open up the pearly gates,
Well there ain't no time to wonder why,
Whoopee! we're all gonna die."

The roots of this User Interface war go deep to an era long before Java existed... the war is really about the best way to implement the client aspects of client-server computing: Dumb-terminal versus Smart-terminal.

I've said it before, and I will say it again, a computer that is not regularly connected to the Internet is not of much use to most people. To quote Sun's motto: "The Network is the Computer"... and that makes the device that you sit in front of a glorified terminal.

At this point in time, I believe that the substance of the battle between JSF and Swing is mostly over the amount of functionality that an application possesses when it is not connected to The Network:

• JSF applications are powerless when The Network is absent
• Swing applications can operate on local data when The Network is absent
  

My belief that The Network is at the root of the war was reinforced on Wendesday when I got a sneak peak at Oracle's ADF Rich Internet Components at the Oracle Developer Day held here in Austin. The newest components are snazzy-looking, AJAX enabled, and with Tools like JDeveloper it is very easy to create non-trivial applications with them.

From a presentation on NetBeans and Matisse presented by Greg Sporar at Austin's JUG a few months ago, I also know that it's pretty easy to create non-trivial applications using Swing components.

If you will grant me the latitude of gross generalization, the biggest difference (from an architect's perspective) between a Swing application and a JSF application lies in where the bulk of the application logic is executed... on the client (Swing) or on the server (JSF).

No doubt we should quibble about the phrase "the bulk of the application logic"... It's quite possible to build a very "thin" Swing front-end on top of Web Services, and with more and more client side Javascript the AJAX components can seem to be pretty "thick". Add to this equation the use of Derby-in-a-browser to provide offline capabilities and I am not left with much of a distinction between Swing and JSF clients.

So what are we fighting for?

I think it is easier to say what we are fighting against... We are fighting against having to use wildly different programming styles in a single application. I say "We", because I think this is the goal of both the JSF and the Swing camps.

Swing advocates obviously like the ability to do everything in Java and they hate markup (except for Josh Marinacci and Ethan Nicholas).

JSF advocates are much more markup-tolerant, but they want to hide all the the markup ugliness (and Javascript) within nicely wrapped packages.

If you can get past these differences, then you will find that the client-side JSF event-handling-logic is not all that different than Swing's event handling logic.

So is either side ever going to "win"? I dearly hope that they both do... and I pray that both sides don't destroy each other or:

"Whoopee! we're all gonna die."

Longing for an Integrated Maintenance Environment

We are blessed with many great IDE choices for developing our code... and now we're beginning to see glimpses environments that will help us maintain deployed applications

When the dot.com boom went bust, I was fortunate enough to land a job as an enterprise architect in the development and production support group of a mid-sized corporation. After a career in R&D, I found myself in a more typical IT environment where keeping things running was way more important than implementing new features.

This change in perspective from R&D to IT was a very good thing for me. After years of delivering solutions to customers and moving on to the next challenge, I was now the producer, consumer and supporter of my own team's creations. I now had to "eat my own dog food" in a production environment (24x7x365).

When you really have to live with (and support) your own products, your perspective on what is important changes.

For the most part, business applications are conceptually simple. You present a form to a user, validate the responses, and process the request. It's hard to conceive what could go wrong after deploying such Simple Business Applications^TM, but inevitably Murphy's Law kicks in at 2 a.m. during peak processing season.

Another common factor of Simple Business Applications^TM is perpetual change. SBA's are living and breathing entities that evolve over time: Fields get added, fields get deleted, validation rules change. Edicts from governing agencies impose new constraints. Changing business partnerships impact workflow. Nothing is truly static.

Add to the nature of SBA's the inevitability of staff rollover (developer's tend to change jobs frequently), and you start to place a premium on code that is robust, easy to fix and easy to learn.

Continue Reading...

Happy Ending to the Netbeans "Replace Dialog" tale

NetBeans 4.1 Update:

Hurrah! NetBeans 4.1 has "fixed" the replace dialog problem that I blogged about last year... Eclipse could limit "search/replace" to the selected text and NetBeans couldn't.

The new "Replace" dialog allows you to limit the scope to the selected text:

Many thanks to the responsible party. It's a small change, but one which really makes a big difference (to me).

Continue Reading...

This work is licensed under a Creative Commons License.