Update I posted the gist of this thread to the SOA and REST Yahoo discussion groups. Here are the threads:
Service Oriented Architecture
REST
##########
After about a week of quiet, the panic is back on REST ("Web Style" according to Tim Bray which I think is better) vs WS-*.
Tim Bray takes the gloves off:
He is preaching to the converted when it comes to "Web Style" vs WS-* for me. Perhaps the acronym "SOA" will eventually meet its demise, but, we'll just have to come up with something else. Sadly, "Web Style" doesn't cut it for every type of service. We've had some flavor of "SOA" since ARPAnet. We like services. And I'm sure they'll be something after "Web Style".
But I have to wonder, just how much Web Style do we need?
Which types of integration is it appropriate for? I'm specifically talking about internal integration within a company. Web Style all the way for integrating between companies IMHO.
I have my own shiny object -- its messaging (e.g., JMS). While I don't think it should be used for everything, it works very well for providing the "plumbing" for complex integration. Yes, JMS is just a Java thing, but good JMS providers have C, C++, COM, .NET, & HTTP APIs so it is platform independent enough to get the job done. I'd love to see a truly platform independent reliable messaging, but as the REST guys say re:HTTP, JMS is a technology that works and is here today.
I like Web Style as opposed to WS-* for the class of service that should be over the web. I just don't see either being that great for extensive internal integration compared to something like JMS that provides things like clustering, fail over, guaranteed delivery, error queues, and durable subscriptions.
As part of the future state architecture planning I am participating in, we are trying to identify the high level types of integration that are common for us. I think they are common for most large companies. Here they are:
Update 18-APR-06: Added some prose around each integration type.
- Sync Request/Reply
ServiceA.Thread1 sends request and waits for response from ServiceB.ThreadX.
- Async Request/Reply
ServiceX.Thread1 makes request. ServiceX.ThreadX receives response.
- Async 1:1 (fire/forget)
ServiceA.Thread1 publishes message and does not wait. ServiceB.Thread1 receives message. ServiceA.Thread1 may or may not be aware of ServiceB.Thread1 (depends on protocol used).
- Async 1:M (pub/sub)
ServiceA.Thread1 publishes message. ServiceB:ServiceN receive message. ServiceA.Thread 1 unaware of ServiceB:ServiceN.
- Async 1:M Request/Reply (pub/sub Request/Reply)
ServiceA.Thread1 publishes message. ServiceB:ServiceN receive message and send response. ServiceA.Thread1 receives N responses. ServiceA.Thread1 unaware of ServiceB:ServiceN - just know that N responses were returned.
- Async M:1 (event stream subscription)
ServiceB:ServiceN publishes message. ServiceA.Thread1:ThreadN receive messages. ServiceB:ServiceN unaware of ServiceA.Thread1:ThreadN
- Batch Feed
ProcessA sends file to ProcessB
I think Web Style is perfectly fine for a certain class of Request/Reply services. For example looking up existing customer information. It definitely wouldn't be my first choice, however, for something transactional like creating an order. Not saying I'd never do it, just wouldn't be my first choice. My problem with using Web Style at all for heavy integration is the coupling and complexity that starts to add up when you have a lot of services all doing Request/Reply. Each client has to handle errors and retry logic. Certainly not rocket science, but it is more then I want to deal with.
I think that Batch Feeds could also be done via Web Style. They don't tend to be coupled sequentially the way I described Request/Reply above.
Once you get to anything async, however, Web Style quickly falls down for me. Lets start with "Async Request/Reply". An example is requesting a report from a third party that takes processing time (say 1 day). You make the request, and want to get the reply async. With Web Style, people often end up making the request for the report, get an id back and then polling at an interval for the results for the id. You add logic so that you don't poll until you think the report is ready, etc. This is complexity. Again, not rocket science, but lines of code with defects in it to be sure. To be fair, you could have the report service call you back which isn't as bad - again, however, there is non-trivial complexity here when you deal with the error cases. If you are writing one service, it is no big deal - its when there are more then a few where you get a cumulative amount of complexity that starts to get your attention.
For "Async 1:1", you can do that with Web Style like Request/Reply. You do have to contend with error cases, however. And "out of the box" you don't get guaranteed delivery. You can't send the message persistently so that you know that the send being ack'd means something. You can certainly do this by hand - and it isn't that hard, but again, this stuff tends to add up and every time you roll your own guaranteed delivery, there are going to be defects. Not the type of stuff I want to deal with.
Alright, so now we are getting to the harder stuff - "Async 1:M", "Async 1:M Request/Reply" and "Async M:1". This is where I really struggle with Web Style. I've tried to challenge myself to find a way to do this with Web Style, but haven't gotten really far. Admittedly, however, I tired fairly quickly - please let me know if you know of something.
How would you do "Async 1:M"? This is classic pub/sub. A client sends 1 message and N subscribers listen to the message. The client has no idea who is listening. I don't know of any way to do this w/o writing a ton of code.
How about "Async 1:M Request/Reply"? This is when you send one message and get N responses. This is pub/sub request reply.
Last one on my list is "Async M:1". This is 1 service listening to N destinations / Topics for events.
Don't get me wrong. I'm all for Web Style. The sooner SOAP and WS-* are deprecated, the better, IMHO. I just don't see Web Style as a silver bullet for integration any more then anything else is. Not saying that Tim is saying that - just think that there are lots of cases in integration where Web Style OR WS-* isn't the best choice.
3 comments:
Hi Mike,
Can you tell us how you determine the content definition of canonical message formats in an EDA?
We at Dutch Railways struggle in finding a method to define canonical message formats to be published. E.g. do we define all possible data items or just the primary keys?
Can you advise us?
Hi Jack,
I actually blogged about this a year ago here:
http://fuzzypanic.blogspot.com/2005_04_01_fuzzypanic_archive.html
My opinion has evolved a bit, however.
The industry standard that I am exposed to the most these days is Acord. They have 2.0 coming out, which, lets users use a small set of it rather then the entire thing. I'm hopeful that this will let you dumb it down more. They are also hoping to support events properly. In past versions there was always a request and a response, which, of course if often meaningless in an event driven system.
In general, I think that it is appropriate to use industry standard XML formats when you are communicating with 3rd parties. But, within your own system, you need to own the canonical message format and keep it simple as possible.
So now to your question. Mind you I don’t know what your requirements are etc. so take this with a grain of salt.
1. Keep it simple – this is not something you want to over engineer
2. I’d avoid publishing the keys and instead send the actual data. Publishing the keys will force you to go back to the data source a lot. IMHO, you want to avoid hitting the data sources as much as possible. If you are using messaging that provides guaranteed delivery/durable subscriptions, trust it. Avoiding hitting the database as much as you can once event is in the system. Part of my life is spent fixing a system that didn’t do this – it isn’t fun. By publishing the data, listeners to the events can do content based routing, scribble on the document and pass it along, etc. Most of your services shouldn’t even connect to a database. They should just listen for events, do their bit, and generate new events. In your design, triple check where you are doing database access – try to limit it as much as you can.
3. Try to use natural keys if possible
4. Natural keys are often not possible and problematic – most people use a UUID/GUID in this case – this works great, but can require some effort you may not have planned for
5. It is ok to put the primary key in the message – just tuck it away in a specific system specific area of the message. The primary key is included for services that need it, but all the data is there so 99% of services don’t use it – its just there when you absolutely need it.
a. The OAGIS spec handled this pretty well, might be good to use as a reference, just beware of using any industry standard too much as they can require a ton of effort and have zero payback
Aleksander,
I am aware of those specs, but thanks for the post.
Based on this:
http://www.tbray.org/ongoing/When/200x/2006/03/24/WS-Evolving
I don't think the tools in the market will be there for a while. And then we'll see if they are accepted etc.
I do hope that there is something platform independent in the future for events. I'm just not sure that it will be WS-*.
In the meantime, I'll keep getting the job done with JMS.
Post a Comment