John Reynolds's Blog

Why do Java developers hate BPM?

Java developers hate BPM.

The preceding sentence is (of course) intentionally tailored to be controversial. People tend to read controversial blogs, and I'd like you to read this one. Now that I have hopefully grabbed your attention I'll tone it down a bit...

A lot of Java developers hate having to use BPM tools instead of the object oriented tools that they are comfortable with.
I work on a lot of BPM projects, and I work with a lot of other folks who work on a lot of BPM projects, and we have all encountered resistance from traditional Java developers. Java developers (in general) would rather use frameworks like Struts and Spring than be saddled with the constraints of a BPM suite.

Java frameworks like Struts and Spring are in the background... they provide just enough support to "set your creativity free" so that you can be a real programmer. You can build almost anything with Spring or Struts (if you've already mastered the intricacies of Java). They are light-weight, they're agile, and they look sexy on your resume.

BPM suites are in-your-face. They rob you of your creativity. They dictate to you how you will develop your application.

BPM suites make programming boring. They force you to use point-and-click and drag-and-drop tools to design your process diagrams, data models and forms. What's worse, they actually encourage Business People to model processes and design forms on their own... Fortunately most Business People are too intimidated to use these tools, but it does open the door for them to look over our shoulders and meddle in our affairs.

That certainly doesn't sound like something that real programmers would like, does it?

I'm not being subtle, am I?

BPM suites are a threat to traditional Java programmers. These suites are far from perfect, but even in their current state we can see where things are heading. The days of the Java Guru as indispensable are fading... We've used Java to build tools that make knowing Java itself less important, and that's opened up competition for us from folks who didn't spend years learning Java.

We're victims of our own success... and that's going to cost us.

That's why Java developers hate BPM.

Comments

• Here is why I "hate" (or just haven't been able to use) BPM: - I still don't understand how I would use them. My business software development tasks have been along the lines of: "build a migration tool for this proprietary system", "build a new set of reports", "build a configuration interface for XYZ", "internationalize this app", "refactor these old screens to do XYZ". I can't honestly see where BPM would ever be a practical tool to get something done.. - NetBeans has a free BPM tool, but it just looks like a simple Web Service automation tool. That is completely useless to the business requirements and concerns that I encounter. Even the more fancy tools sound like fancy high level process scripting tools which still don't offer much value. - None of the good BPM suites are freely available to devs so it's hard to experiment. They cost a lot of money and my boss isn't buying them. - Where are the success stories? I've seen tons of amazing software with killer features including competing enterprise software. I also look at the tech and tools they use to create their products, and I never see BPM. To me, BPM seems like an abstract process tool with limited applications. If I mentioned to my boss, he would think pedantic and useless academic nonsense. My ears are open: show me some real world applications of this technology.

  Posted by: giesen on October 10, 2007 at 07:01 AM

• Hi, I'm a Java developer and sure, I hate BPM. I agree with your 8 first paragraphs, but disagree with your conclusion. I think that finally, we hate BPM cause as you say, their idea is that Business People do the modelling task, but due to the fact that Business People doesn't use it, we end doing it. We hate BPM cause we shouldn't be using it. If I had to make spreadsheets cause some Business Person doesn't know how to use spreadsheet software, I will hate it for sure, but not because I think that spreadsheet software makes me less indispensable. Greetings (and sorry for my poor english)

  Posted by: patejava on October 10, 2007 at 07:02 AM

• patejava,

  You make a very good point... Java developers are not the intended audience of BPM tools. They are focused at a hybrid between a Business Analyst and a Programmer... and that hybrid is very rare. Business schools don't generally teach the skills necessary to design a good process or a good form, and Programming schools tend to drive away students who are interested in making businesses run well.

  I think we're going to see more of these hybrids in the future, but at present we do have a lot of "way overqualified" folks using these BPM tools.

  -JohnR

  Posted by: johnreynolds on October 10, 2007 at 08:04 AM

• giesen,

  I feel your pain. It's very hard to understand the usefulness of a tool unless you have an opportunity to use that tool on a project which needs that tool.

  Netbeans isn't really a BPM tool... it's a BPEL tool. BPM combines Human Powered services with automated services, and the Netbeans "BPM" tools just deal with automated services..

  Intallio has a free BPM suite that I hear is pretty good... but most BPM suites are very expensive.

  As for BPM success stories, you just need to look again and you'll find them. Thousands of them.

  Thanks for sharing your thoughts,

  -JohnR

  Posted by: johnreynolds on October 10, 2007 at 08:17 AM

• I'm with giesen. We just haven't seen the need for such a tool yet.

  In the RDBMS world, we have the concept of normalization. Everyone who's ever done anything real with a DB knows what this is. In Java, we'd call it "refactoring". Taking the redundant bits out of the schema or code. As you progress through the normalization process, your schema progress through different "normal forms" (first normal form, second normal form).

  In complicated schemas, on dynamic systems with a lot of loosely defined configurability, you almost always end up with some kind of "name-value" pair table. The ever popular "property" table.

  A next step would be where you have some kind of generic many-to-many style join table that can literally connect "anything to anything". Allowing arbitrary hierarchies and graphs of elements within the schema.

  Around here, we've coined the term "ThingThing" table for when we see these kind of structures show up within a schema design. When the ThingThing table has reared it's ugly head, we've basically reached "the absurd normal form". At this stage you're writing a DBMS on top of a DBMS and need to stop and think about what you're doing.

  As an old school, die hard, "started back when all we had were zeros" grognard programmer, I've seen the rise of abstraction in coding all the way up from keying in binary codes through the front panel of a computer to where we are today.

  And when I look at some of these new systems, like the BPM systems, I see what I'll call, with respect, "ThingThing" containers. Infrastructure that binds together Infrastructure.

  Now, we've had a form of the ThingThing container for almost 40 years now. Anyone who's typed "ls | more" in to their shell prompt has "bundled services with a common interchange within a managed environment". The Unix command line is the quintessential SOA environment. thing | thing | thing.

  Now, while apparently humanities majors can master the Unix command line when focusing on the task of creating dissertations using the document processing toolchain surrounding TeX, I haven't seen many business analysts who were particularly comfortable with it. (No doubt they exist, but they've historically managed to avoid my attention.)

  Of course, beyond a few one off overnight batch processes, or sys admin automation scripts, most organizations do not as a rule leverage something like the Unix command line for their fundamental processes. It's expensive and inefficient. It's also cryptic and user hostile. Even the best of us get queasy when we see a command line with more than a 1/2 dozen or so | in it. Just gets unwieldly.

  But what you end up discovering, with even the best of humanities doctoral students, is that inevitably, stitching together coarse services inevitably isn't flexible enough. The services are simply that: too coarse. Or the workflow becomes so complicated that a few pipes on a command line no longer do the task properly.

  Then you start getting in to simple branching, and iteration. Then you get in to keeping track of state between service invocations in order leverage earlier work, or make decisions later on in the process.

  The student then most likely starts putting that process in to notes in their notebook, or on a sticky on their monitor. Some may even advance beyond the command line and try recording their process in to a shell script, which is a much better environment for these more complicated processes.

  Then their process starts getting complicated by system limitations: need more disk space, or more memory, or the file server goes off line.

  Certainly, no scripter is going to handle those scenarios. They'll let the processes simply fail loudly and violently and punt back to the Sentient Lifeform as the ultimate exception handler.

  But the point is, that what on the white board may start as a simple process inevitably has the potential of becoming more complicated when it starts hitting the real world and real systems.

  The reality is that even given the most simple of processes, those processes actually run on a computer. And the computer is the most cantankerous, pig headed, and brutally honest machine on the planet that only understands "do what I say", not "do what I mean".

  No matter what the diagrams wireframe says, there has to be a fundamental understanding of the realities that those little blocks it's wiring together represent. There are a LOT of very subtle details contained within that "Post Invoice" bubble, and try as we may, those subtleties WILL stick out and affect those around them.

  In the end, the folks that need to create those diagrams need to understand computers and computing. These folks ARE programmers, and programming requires a specific kind of mindset as well as technical knowledge.

  That Intallio site has a line (paraphrasing) "One of our boxes represents 10 lines of BPEL which represents 100 lines of J2EE code". So, while it may only show up as a single box on the diagram, you really had better have a good idea what's happening in those "100 lines of J2EE code". The complexity is still there, no matter the color or shape of the little box.

  Obviously abstractions have made all of our lives better in the long term over time. From binary codes to assembler mnemonics to C to Java. "One line of Java represents 10 Java Byte codes which represents 100 machine language instructions.". But the best Java programmers have a good idea what those 100 machine instructions are doing. And the novice programmers are at times punished by their ignorance of them.

  I've yet to see a high level abstraction on a computer work well with a layman. I've yet to see a domain expert do well when working with something like a Domain Specific Language (which is all that something like BPEL is, frankly, though even it is too general). Every time we've tried to push back in to the user base to empower them, in terms of internal business processes, they inevitably push back on us because of the impedance mismatch between what they know and how they see the world, and what the computers world is and how it wants things done. So, inevitably, we programmers get hauled back in to act as the interpreter.

  I've spent many a long hour writing routines in the scripting language that I designed, with the help of the end users, for them to use, that they inevitably abandoned because the systems are too complicated, no matter how you sugar coat it for them. The subtleties too arcane. "I'm a Doctor, not a CSEE professional!"

  As a developer, I look at the "ThingThing" containers and puzzle as to "when does my system become big enough that's it's worth throwing this blob of infrastructure in the middle of it all -- that I need a scripting language interpreter and runtime that needs 1GB of RAM all for itself". I haven't got there yet. I haven't go to the point where it seems the costs of knitting together our code using these large runtimes and marshalling the data in and out of the different modes and boundaries is cheaper, more efficient, or more productive than creating a new Java method, or adding a few lines of code to an existing one.

  Obviously systems like that exist, but I don't know any that are not maintained by a legion of specialized administrators and coders rather than business analysts and domain experts.

  When the analysts and domain experts get to maintain and build their own systems, then they can use the tools they want. But right now we're doing it ourselves, with their input, and that's why we push back against these kinds of tools.

  Make no mistake. I am all for empower the end user and shifting the as much of the development burden on to them as possible. I raise up on the pedestal those ad hoc programmers fighting Excel, MS-Access, Hypercard, and any other high level "see, even end users can use it" tool to make their lives better. I WANT an army of coders and "do it yourself"ers. As an example, I don't think there's a construction contractor out there that sees the Home Depot's of the world as a threat to their livelyhood because they offer how-to books and supplies to empower the novice to install their own sink or drywall. And it would be interesting to someday see myself code my way out of a job. Maybe I can open up that bakery then.

  But it's not happening, and I haven't seen some great shift happen yet. Computer are still computers. Bone stupid powered sand castings that are yet the most inscrutable and difficult to work with entities on the planet. I love programming, but boy do I hate computers.

  When the better computer comes along, I'll worry then. But until then, I have no fears. And I want to use the tools I'd like to use to get the job done quickly, accurately, and efficiently.

  Posted by: whartung on October 10, 2007 at 11:45 AM

• I used Lombardi's BPM suite at my last job for about 1 1/2 years, so I think I can speak to this from experience.

  The reason we disliked working with your BPM suite is because the developer's tools sucked. Why? Go ask you engineers to write the next release of your product as a flowchart in a visio-like tool. They will be limited to using javascript. They will have no access to an automated regression testing system . They will have none of the features of eclipse or Intellij Idea they have come to rely on. There will be no build deployment scripts, deployment will involve copying files manually. Oh, and they will not have version control.

  I'm curious what they would say. Ask them how productive they will be.

  You also made some comments about java developers wanting to use spring and struts to pad their resume while indulging their desires for creative freedom. You insinuate that developers want to use these tools for selfish reasons, rather than using the tool or product which creates the most value for the business.

  There's some truth to that. I think that many non-technical businesses have come to believe this, creating distrust between the developers and the business side. I know my last company had this problem and I imagine lots of vendors and sales teams exploit this.

  Also, you mentioned Spring and Struts, I assume you mention those because they provide only a fraction of what your BPM suite provides. Well, you could also include quartz to give you a scheduler, you could include lucene for search, and you could include jBPM or OsWorkflow to provide a workflow engine. There are any number of other open source modules you can plug in to quickly add features. Of course there is developer time involved here. But, at my last job we spent countless hours customizing our BPM installation because by golly, those pesky business users are never quite happy with an off the shelf product. Many a time I wondered if we would have bought the product if we had known how much time and consultant dollars we were going to spend customizing it.

  Also, as far as success stories go, I don't know how much faith I'd put it them. My company was listed as a ''success story'. The reality was that the project was a disaster, the end users were up in arms, and as of last march the entire development team had moved on.

  Posted by: robert_perkins on October 10, 2007 at 12:24 PM

• robert,

  I apologize for any "snideness" to my Struts/Spring comments.... I think that we all pick our tools for selfish reason. We pick the tools that appeal to us. We pick the tools that make it pleasurable for us to do the kind of work that we love to do.

  Spring and Struts are great for builidng things like BPM suites.... All of the BPM suites that I know of heavily leverage things like Struts, Spring, Quartz, Lucene, etc.

  The kind of work that is involved in building a BPM suite is not the same as the kind of work that is involved in implementing and managing business processes. One is primarilly focused in developing infrastructure, the other isn't.

  I use BPM tools for purely selfish reasons. I once worked on a massive project involving dozens of engineers who worked for over three years to implement what was actually a rather simple loan origination process. Prepare Application, Review Application, Dispense Funds, etc. Literally millions of dollars were spent on this project, and at the end of the day we still couldn't effectively monitor the progress of an application through the system. I recently developed a similar process with 4 people in 6 months using a BPM suite.

  The solution was not "perfect"... The user input screens were positively boring and response time wasn't as fast as I'd like, but in terms of a maintainable and trackable process we were much further along than my previous experience with a totally custom solution.

  Why the big improvement? No magic involved, we simply started from a point where the infrastructure was already in place rather than building from the ground up.

  I pick BPM tools because (at this point of my career) I enjoy building and improving processes... and I enjoy the idea of empowering Business People (like my dad) to build and improve their own processes.

  As you point out, BPM suites aren't quite ready for my dad yet, but I can envision them evolving to that point... I cannot envision traditional Java tools ever in the hands of any but skilled programmers.

  Thanks for sharing your thoughts...

  -JohnR

  Posted by: johnreynolds on October 10, 2007 at 04:03 PM

• whartung,

  Great response! Thanks for sharing your thoughts.

  -JohnR

  Posted by: johnreynolds on October 10, 2007 at 04:04 PM

• I've been doing java development with a big bpm solution for a couple of years now. An issue we had was that we were not able to migrate our production workflow to qa and dev the same way we could our db. That made testing and recreating production issues difficult. Even though our tool supports different versions of a map all running at the same time we never hooked that into our release versions to be able to quickly deploy an entire environment at a certain version number. For me the bpm tool was worse than a database in terms of being something that you could not stick in source control and hook into automated regression testing. It was the part that always got mocked up. Maybe if there was something like an "in mem" workflow like hsql, with something like hibernate/dbunit that could take "the" workflow description from source control and initialize it for the duration of a test ...
  When sql came out people thought that a csr would be on the phone getting an address update from a customer and typing "update customer set address = .... where id ...." while the person was still on the line.
  When rules engines came out people thought that BAs would change the rules and redo the application while it was running w/o need of a developer. I think the same thing has happened here with bpm. I don't think it is a bad idea but it has to be easy for developers to use. It has to fit in with whatever process a shop is running. Having to get a license to run the admin gui to set something up is not O.K..

  Posted by: johnwatson on October 10, 2007 at 05:47 PM

• johnwatson,

  There's still a lot of ground for BPM tools to address, that's for sure.

  My personal grail is for easier "In-Flight" process changes... beyond simply modifying process flow we need easier ways to transform process data from one version to the next on the fly.

  I think the next few years are going to see huge strides in these areas.

  -JohnR

  Posted by: johnreynolds on October 10, 2007 at 06:23 PM

• BPM has two main problems, both of which have been raised already, but that I would like to emphasize.

  One is release management; many expensive products suffer from this problem. You can't "check in" your application to (your own) version control system and reproduce a working deployment with a single button push. Reproducing, verbatim, what exists in a development environment in a production environment seems to be a problem that never occurred to these vendors.

  The other problems is that these systems give a Lil' Tykes toolbox to the business folks so that they don't hurt themselves. But the suits just turn around and ask Daddy to build the tree house, pitching a fit if he pulls out the Porter Cable tools. That's why I "hate" BPM.

  Posted by: erickson on October 11, 2007 at 10:10 AM

• JohnR, you made a point earlier that needs to be emphasized. You compared two experiences you had with building a BPM-based application, one which was horrific, and one which was very smooth. I believe the difference points out that you can't build a business process in a vacuum. If a business analyst starts out defining an application from scratch, they will produce a horrible mess. However, if the technology group implements a well-defined infrastructure of business-aligned services that are clearly divided into entity, task, and utility services, then the analyst will most likely be able to define something better. To some extent, I believe that this situation is exacerbated by the idea that business analysts can use BPM and BPEL solutions to magically produce a working application. It's perfectly fine to give them tools that allow them to specify their needs in terms that technology can understand, and can more easily translate into implementations, but that step is important in the development process.

  Posted by: dkarr on October 11, 2007 at 10:16 AM

• We're victims of our own success... and that's going to cost us.

  The first time I heard this was as an intern, about 1995 I think. Back then it was case tools.

  Posted by: tcowan on October 11, 2007 at 12:17 PM

• Me, too. I prefer expressive dynamics like forte, mezzo-forte, fortissimo, pianissimo... but BPM makes mixing easier -- viva Grandmaster Flash! Seriously... my point? Who cares? It's a tool. If you don't like it, don't keep it in your toolbox.

  Posted by: evan_j_goff on October 11, 2007 at 01:02 PM

• evan,

  The BPM tool is not in your toolbox. It's in the toolbox of the company that you want to work for... and they're requiring you to use that tool.

  Does that change your perspective a bit?

  -JohnR

  Posted by: johnreynolds on October 11, 2007 at 01:32 PM

• JohnR, Many of the other commenters seem to share the same concern that I have with BPM tools: Configuration management (by which I mean version control, regression testing, and maintaining different configurations for test and production).

  When you're sliding backwards all the time, it doesn't matter if the tool helps you to change fast. I've been forced to use tools that made us able to fix small problems easily the first time, but that made it impossible to manage those changes and roll them into production in a controlled fashion. Now, truth be told, the same thing holds for many J2EE servers.

  What is your experience in this area? Does your favorite BPM tool allow for good (I would even settle for decent) configuration management?

  Posted by: jhannes on October 11, 2007 at 01:59 PM

• I have worked with other BPM tools, and thought I would post my thoughts. First off, I can admit that BPM projects fail too, and fairly often! I don't think it's any huge revelation. They fail for many of the same reasons that regular application projects fail - poor timelines, poor business analysis, poor scoping, etc, etc. And of course BPM tools add some pain to the process too, as was mentioned - learning a new tool, struggling with release management, and battling the "just build a diagram, install it, and go!" sales pitch that gets thrown about. BPM is not for every project. As John mentioned, you use this to build out BUSINESS PROCESSES! How do you recognize them? Well, they should have people doing some stuff, and the system doing others. The people do the stuff that they're good at, that they are comfortable doing, and that an application designer probably shouldn't try to code for. The system takes care of some of the plumbing that John talks of - tracking progress, auditing a process, automatically routing work to people, handling common business issues such as approvals and escalations - if you can't find these aspects in your application, then move on! But if you do decide to use BPM, I think you'll appreciate all of that plumbing that you get. And I prefer to leave as much complexity out of the system as possible. The nice thing about BPM is that you can always count on people being involved in the process, and people are very able to handle random or arbitrary fluctuations in a process that I think are better to avoid coding for. If you can do so, you'll find BPM projects to be less painful.

  Posted by: jnedumgottil on October 11, 2007 at 04:03 PM

• whartung,
  
  That's the most intelligent post I've read in a long time. Thanks for expressing my same experiences/options so clearly.
  
  -Bryan

  Posted by: prime21 on October 11, 2007 at 09:34 PM

• People have been saying that "within a few years" there will be no more need for programmers, that the business people will just pipe their requirements into some piece of software and it will pump out a fully functional application that works perfectly and is a dream to use a few minutes later for at least a decade. It's not happened yet, and I doubt it will ever happen.

  What happens is that the creators of such "tools" just invent another "programming" language in which those requirements have to be expressed, so someone has to do the programming of that (and it won't be the business people, as they lack the skills and will to do so), and the system pumps out a skeleton of an application that needs massive customisation to work at all, let alone do the task required and actually be usable.

  And there's why programmers hate such tools, it relegates them to cleaning up the mess of yet another code generator, and one that's outside their immediate control at that.

  But for Dutch programmers there's a major reason to hate BPM: it's a 40% "luxury" tax on all new cars we have to pay here, making buying or leasing a car extremely expensive.

  Posted by: jwenting on October 11, 2007 at 11:00 PM

• John Reynolds's Blog Why do Java developers hate BPM? Preview your Comment BPM (or, more appropriately, workflow) is simply a checklist of 'things' that need to get done for long running processing (say, for example, a fulfillment, or a insurance claim, or whatever.) if you have a 1-state 'action' like a search (ie, you put in input, and it spits output, and then you're done) then you don't need workflow. BPM has nothing to do with spring / struts. The fact that many tools do allow you to model your process using snazzy drag and drop is an aside. Many also let you model it in code. The benefit is the same: if you know what you're inputs and outputs are for each step of your "business" process, then programming the steps along the way becomes easier because each "step" is decoupled from having to know about what's happening downstream or what happened upstream. You see? Suddenly programming a huge system with many actors becomes a matter of coloring in the lines. Also, don't confuse BPEL with BPM. BPEL is a pretty sad excuse for a BPM standard. It completely forgot about the human task-list component typical of most "business" processes. -Josh Long

  Posted by: starbuxman on October 12, 2007 at 01:31 AM

• I guess I am going to have to hit Google to figure out what the heck BPM is...

  Posted by: mjparme on October 12, 2007 at 07:18 AM

• I think it's important to remember that BPM defines "what" should be done, and in which order to do the tasks. The actual, how to do each task, is much more in the Java and OO programming domain. You should build your low level "services" using Java/Spring/Hibernate etc. because these tools are much more suitable for this. Also, I definately agree with jnedumgottil that you have to actually see if you have a complex Business Process before deciding if you should use BPM in any form. If you are building a more or less stateless system, there's no point in using a state engine... and BPM Engines are state machines. However, if you need to coordinate automatic and human taks and need auditing, process monitoring etc. then BPM may be a good idea. Also, many BPM tools such as the ESB's BPEL engines have very good integration infrastructure, that makes it much easier to weave your services (or tasks) together.

  Posted by: mario_aparicio on October 24, 2007 at 04:23 AM

This work is licensed under a Creative Commons License.