Saturday, May 06, 2006

EDA Lessons Learned - Canonical Message Format

See EDA Lessons Learned for the list

I posted about Canonical Message Format a little over a year ago. I still agree with most of what I said then.

I have been exposed more to the Acord insurance standard since then. Sounds like Acord is addressing a lot of my complaints in the 2.0 version. Main quote that caught my eye is:

V2 will be delivered with an option to "slice" or prune the schema down into smaller files with the message content of your choosing.

That says a lot on how "standard" the XML really is on the wire in the insurance industry. This is essentially admitting defeat IMHO. Nothing wrong with it - just facing reality. The truth is that every carrier has differences - you make a valiant attempt to map to the standard, but you still have 20% more to go. There is an extension mechanism to account for this.

The trouble I have with industry standards like Acord is that if you follow them without pruning the original schema and making it your own (i.e., owning your contract), you have a very loose contract between services.

I'm all for using the industry standard for integrating with third parties. I think it is ok to use internally too, but it makes me nervous. People tend to toss the "standard" term around and act like the fact that it is being used as the canonical message format means something major - as if you are going to get a huge payback for it. I have never seen anyone get a big payback from any XML industry standard. And it takes a lot of effort to follow the standard. New versions come, the company has extensions, etc. At some point, people throw in the towel and have something that looks like the standard, but the chances of it resulting in any sort of out of the box interop with anything are nil.

So I think its mostly about setting expectations ... it is ok to use an industry standard as your canonical message format. I think it is wise to go into it with the "prune" strategy. Just use the standard as a reference. Don't event attempt to use the standards schema files. Just lift the structures. And dumb it down as much as you can to make it a tight contract. You will stray from the standard eventually - better to just admit defeat going into it IMHO. No shame in that.

Here are some more thoughts:

  1. Avoid request/response designation in your root elements.

    EDA is not sync - don't act like it is. Even if you have some workflow that is sync, don't bake it into the messaging model - it will likely change over time. The request/response in the XML will just confuse people.

  2. If using XML schema, keep it simple - avoid the fancy stuff
  3. Question elaborate tools that make managing your canonical message format easier

    These tools often make it easier for you to make things more complex then they need to be and lock you in. They typically require training - so only a couple people know how to use it (single threaded through them). Your canonical message format should be simple & should only require something like XML Spy, source control, and a web site for easy reference.


Sarge said...

Just lift the structures

That's exactly it, bro. You're never going to get out-of-the-box interop so just give up on the pipe dream. But if you're structure is right, you can map to any of the schemas you encounter. Remember the clowns we used to work with that stored the telephone number as one big string of digits? They threw away the structure and then got screwed when they needed to separate area codes from the rest.

Architecting a message format is like any other kind of software architecture. It's easy to do too much and it's easy to do too little. And like other forms of software architecture, it's extremely sensitive to the talent of the person doing it. It's one place you need to make clown-proof.

Doc said...

However, one thing that "industry standard models do" is establish an "enterprise-wide" perspective on the XML schema...and it avoids the whole "granularity" issue, by defining the canonical message as a "whole". Sure, you're going to customize it...everyone does...but your original benefit is using it to define a message schema that changes very little over time, as apposed to one you try to "reinvent" based on use-cases trickling in.