Michael Champion's Blog

More WS-* specs, more questions about architectural viability

It's as regular as the seasons: as the leaves start to fall from the trees here in Michigan, more web services specifications flutter down from WS-IvoryTower, and more hunters take up their rhetorical shotguns to blast at them. This year it is WS-Transfer and WS-Enumeration , Microsoft has a new web services architecture vision document to accommodate them , and a number of people (Tim Bray is perhaps the most visible) have already wondered why we need Yet Another web services spec that appears to do something the Web already does.

I'm in somewhat of the same quandry that Tim is in, since there are an awful lot of smart people on each side of the fence here. (I recall a W3C Web Services Architecture working dinner when I was the only person at the rather long table without a Ph.D!). Likewise, I agree that the WS-* stack is monumentally difficult to keep track of, that there are some very visible success stories for "web services" that don't use WS-*, and that "design by committee" is not likely to be successful.

But one thing Tim says really helped me to understand why there are such diverging opinions among smart and well-informed people. " I’m deeply suspicious of multiple layers of abstraction that try to get between me and the messages full of angle-bracketed text that I push around to get work done ". Ahh, I think I see it now -- the "XML and the Web suffice" people push around XML to get their work done, and don't have the pain that WS-* tries to cure. The "WS-* is needed" folks, on the other hand, are working on the behalf of people to whom this is more or less unthinkable. "Real" people building enterprise applications of the sort that the WS-* specs target work with objects, databases, transaction monitors, and reliable message queueing systems .. which they need to make more accessible via HTTP *gateways." They don't have the option of simply exposing these systems as stateless Web resources with whom one exchanges XML respresentations, without a massive amount of code rewriting or adapter building, and there is nobody writing books or selling tools to assist them. There are, however, reams of whitepapers, articles, and books explaining how to apply the WS-* approach to the problem, and a considerable amount of success to show for all this.

On the other side, I don't think that very many of the people pushing simple XML over HTTP really have to solve nasty enterprise integration problems to earn a living. (Sean McGrath is a notable exception, by the way, but I recall (but can't find the link) that he on record as noting that the problems that WS-* addresses are real even if solutions that have been built by committee are not likely to actually work). So, I'm hypothesizing that the WS-* bashers are those who find XML and Web technologies the sharpest tools in their toolbox, and think of the problems of linking them with enterprise transaction systems to be an implementation detail; the proponents are those who have to expose enterprise systems over the web, and think of HTTP as just another transport and XML just another serialization. It's not a matter of one side being right and the other wrong, it's a matter of which tools are needed for which jobs. Tim Bray's point about Amazon, eBay, etc. not needing the WS-* stuff to get their job done is well taken, but it's also quite clear that these were built from the ground up to work with the Web, whereas the fertile ground for WS-* are the enterprise systems that were not designed with the web in mind.

What I'd really like to see here is the application of a well-known conflict management technique in which each side has to state the other side's position, to the other's satisfaction, before discussing the disagreement. For example, the new Microsoft web services architecture document praises and aligns itself with the Web in its introduction:"Web services take many of the ideas and principles of the Web and apply them to computer/computer interactions. Like the World Wide Web, Web services communicate using a set of foundation protocols that share a common architecture and are meant to be realized in a variety of independently developed and deployed systems. Like the World Wide Web, Web services protocols owe much to the text-based heritage of the Internet and are designed to layer as cleanly as possible without undue dependencies within the protocol stack."

So, why don't the Web standards suffice for computer/computer interactions? People have been talking past each other on this topic for years now. How about it? Maybe Mark Baker could re-state his understanding of why Don Box thinks they don't ... and vice versa ... before wrapping this permathread around the blogosphere one more time.

At a minimum, I'd like to see documents such as the Microsoft one address the well-known critique that the Web specs already hit the 80/20 point, and be much more specific on why they think these can't be simply extended to handle computer/computer interactions. I'd also like to see the RESTifarians explain and give examples of how they think goals such as "autonomous services" and "managed transparency" can be achieved without building something like the WS-* framework.

Comments

• But is that really the problem?
  My latest 'brilliant insight' into development is that there are some people who take simple things and make them more complicated.

  I guess their position is based on logic that goes like this:

  "I am a software programmer, I produce software. Software is complicated. Therefore my job is to create complicated things."

  Web frameworks are a classic example. Eg the progression:

  Servlets < JSP < Struts < Velocity < Spring

  Till you get to the point where you are no longer programming Java, but instead spend all your time configuring XML.

  I'm the most extreme advocate of simplicity in programming that I know. I'm *also* the most successful programmer I know, and my standards of success are much higher than most other programmers I know.

  Personally, I think everything to the right of Servlets in that list is a waste of time.

  In fact anything that has to be configured using XML, and which is supposed to replace some programming task is really really missing the point.

  If I roll my own X, it will do the job and probably only have stuff specifc to the problem at hand. If I use a generalised framework that 'solves' X for me, then it probably has to be configured for every possible variation of X. Not to mention Y and Z, which are not even related to your problem, but you have to configure for them *anyway*.

  A generalized solution is *much* more complicated to write than a specific solution. So what happens is that when you come to configure your framework, you end up doing a *lot* more work than it would have been to just 'roll your own' (in this case, just wear the 'cost' of the couple of extra keystrokes to turn your lines of html into Strings, and do your string manipulation in the servlets).

  But wait! It gets worse! Because these frameworks do their configuration in XML you instantly lose the benefit of having a compiler spit the dummy when you get your names wrong.

  Whereas with a framework, you need to do extensive regression testing just to find out if the names are wrong. And you need to keep doing it. So you end up having to do a lot of unit testing, so you need a framework for that, and then you need a framework to support the testing (eg mock objects), and then the whole mess is a pain in the butt to build so you need to configure an Ant script or two (or ten), and the batch files to call the Ant script(s) (...think about that one for a second, that'll bake your noodle...), and then you've got a bunch of different frameworks configuring their properties with XML, so lets slap in a meta framework (XDoclet) to meta configure the XML for us...

  ... before you know it, you ain't in Kansas anymore Toto ...

  ...And it takes a team of programmers two months to produce 5 screens of a web application, instead of the ~5 days it would have taken one person if you'd just slapped the #$%& htlm in a freaking servlet.

  There are some really really core things that programmers do:
  String manipulation
  Math functions
  Insert/Update/Delete/Select on database tables.

  And these things are not rocket science, and they are well understood, and we have been doing them for decades... and yet the people who like complicated things are trying to 'save' us from them.

  Whenever I talk to other developers/architects about the advantages of using JDBC/Servlets they will concede the point, but it seems that they think that doing string manipulation and SQL queries is 'drudgery' that we need to be saved from.

  Newsflash: I like programming. I like programming a heck of a lot more than I like staring at pages and pages of densely packed uncommented xml...

  Posted by: rickcarson on September 20, 2004 at 06:51 AM

• But is that really the problem?
  Sometimes figuring out the framework can take more time than just writing code. But you are arrogant to think that all frameworks are a waste of time.

  My team is currently working on a large JavaServer Faces based web application. We put in our dues to get the framework up-and-running for us. But now we are reaping a lot of benefit from that: JSF does validation, conversion and data binding. Those things take time, I know because I have done it the hard way in the past. JSF has saved us a lot of time and will get us a better product to market sooner and cheaper.

  Posted by: jessewilson on September 20, 2004 at 09:18 AM

• But is that really the problem?
  No, thats not the arrogant thing.

  In all likelihood I could have outproduced your team.

  What *is* arrogant, is to say that I could probably have taken a junior developer and in half an hour showed him or her how to take the html pages and slap them into servlets, and then just set them up and they could have given your team a run for their money.

  Where *you* are arrogant is that (a) you assume all clients are willing to absorb the cost of you doing this, and (b) that you can blithely throw out the advantages of a compiler. (!!!!)

  My point which you validate is that it takes longer to use the framework... even though it is not 'the hard way'.

  If it is not harder, then surely it must be easier?

  Yet... if it is easier, why does it take longer???

  And that is where the logic of making things more complicated than they need to be comes unstuck.

  I'm not anti-framework per se. Servlets are a framework, RMI is a framework. In both cases those frameworks are rock solid and simple to set up and use. And yet they have both spawned and unholy brood of frameworks which try to replace them, and the replacements always seem to be much harder to use, and harder to configure.

  It just doesn't make sense. There must be some kind of psychological factor that compels the lemmings who use these frameworks to inflict the pain upon themselves.

  Posted by: rickcarson on September 20, 2004 at 02:59 PM

• But is that really the problem?

  
  String manipulation
  Math functions
  Insert/Update/Delete/Select on database tables.

  
  And these things are not rocket science, and they are well
  understood, and we have been doing them for decades... and yet the
  people who like complicated things are trying to 'save' us from them.

  
  I think the point is that people want to move on from looking at
  problems as just about String manipulation and want to deal with the
  problem at a more appropriate level that hides the unneccessary
  detail. I mean you aren't coding sockets and assembler are you?

  
  Sure, sometimes peoples efforts are misguided and create a more
  complex beast that doesn't actually help. But often they do capture
  the problem appropriately and simplify things; the cost for you is to
  learn the new declarative "language" for that problem domain. Raw
  servlets and JDBC just don't cover validation, layout, caching and
  other problem domains.

  And sure it's also not entirely elegant; maybe if we were coding
  Scheme many of these declarative frameworks could be expressed within
  the language without external XML files etc. But given the reality of
  it all I'm pretty pleased with how XDoclet, Spring, JSP, Tiles etc
  help me out.

  As long as people are programming they are always going to be
  looking for ways to make solutions look simpler and more like "Saying
  what you mean"; except for the Perl programmers who will make
  solutions look more like problems. So this drive for ever higher-level
  specs is going to go on for a long time yet.

  
  

  Posted by: arcturus on September 20, 2004 at 03:07 PM

• But is that really the problem?
  It just doesn't make sense. There must be some kind of
  psychological factor that compels the lemmings who use these
  frameworks to inflict the pain upon themselves.

  I've got a build with all of the technologies I use integrated in, I
  can reuse it between projects fairly well unchanged so the cost to
  start using these technologies is pretty low for me.

  
  When I write a model object the schema and validation code for that
  object is automatically generated with XDoclet, I find this convenient
  and labour-saving rather than painful.

  You just can't talk about this sort of thing in general terms can
  you? Noone seriously believes that there is no progress left to be
  made or on the other side that every framework is brilliant. The only
  sensible conversation concerns specific technologies.

  So the specific point regarding the compiler is definitely valid but mitigated somewhat by XMLSchemas and a good IDE that will tell you that your XML is poorly formed before you deploy anything.

  Posted by: arcturus on September 20, 2004 at 03:22 PM

• But is that really the problem?
  So the specific point regarding the compiler is definitely valid but mitigated somewhat by XMLSchemas and a good IDE that will tell you that your XML is poorly formed before you deploy anything.

  But is that true? A schema and a good IDE plus binding tools may at best tell you that your code is now validly compiled in such a way that it can handle a document that adheres to a schema as it existed at the time the program was compiled. But beyond that, you have the problems that every programmer in the history of programming has had... the problem of the schema evolving, or even worse forking to the point where the various pieces of your application are now out of sync with either the schema or each other. It happened in COBOL with its data structures, it happened in C with structures that were serialized, it happens in the relational database world, and it happens every bit as much with XML.

  In a web services environment, this is even further complicated by the fact that often times, the disparate pieces of the overall application aren't even under your control, or even worse, the schema isn't under your control. I would venture to say it may happen more often, and the results are even more catastrophic. Layers and layers of useless messaging infrastucture do nothing to solve this fundamental problem, and I would argue they don't really do much aside from add more bytes to the sizes of the streams that are transfered.

  To me, XML-RPC 'borders' on progress in that it's relatively easy to write a processor that will invoke your methods for you. SOAP... not very progressive. It did what XML-RPC could do, but made it far more complex. Beyond that, it added what XML over HTTP could do, but also made it far more complex. And as new specs continue to be heaped onto SOAP, we continue to ask 'where does it end?' and 'who does it benefit?'. Benefits the W3C Working Group I guess, because sitting in a room, dreaming up poorly designed specifications based on very little real world experience or practicality is what their companies pay them to do.

  So how is this better?
  

  Posted by: tbradford on September 23, 2004 at 09:38 PM

This work is licensed under a Creative Commons License.