I generally shy away from throwing down phrases like impedance mismatch as they tend to get abused, but no joke - batch and EDA is one.
We deal with this with our large systems (big iron), which are batch oriented.
Bridging the batch world with the EDA world requires careful thought and design. We found it effective to have very clear control boundaries between the two.
We have batch jobs that run in the middle of the night that generate many events. The events are not generated at the same time every night - it depends on how long the batch cycle takes. So basically the EDA side gets huge bursts of events in the middle of the night.
We see the most benefit from Staged Event Driven Architecture when batch jobs produce large bursts of events.
We also have services that want to write to the mainframe when it is in a batch run. The mainframe is unable to accept the writes when it is in this state. The "Rollback and Pause" concept is very helpful in cases like these. Basically, the service (filter in our case) just holds the event by rolling it back to the messaging layer and letting it retry at an interval. Having this logic in a filter rather then the service responsible for writing to the mainframe is good because you don't muddle the service impl. with environmental issues.
Operationally, the batch/EDA impedance mismatch is definitely our biggest challenge. It generally works, but we have our days. Async send from the batch system, clear control boundaries, "rollback and pause", and ErrorQAdmin, make it tolerable.
3 comments:
I have my roots in the old IBM370 JCL. What I like about EDA is that designing an EDA is very similar to designing batch job flows. Where you write a record to a file, you publish an event. Reading from a batch file is consuming from a topic. Where you read multiple files in balance, you come to complex event processing. Batch programs are decoupled. Programs write records and others read records, without any knowledge of each other's existence.
Simply said: Just take out the read/write-to-end-of-file loop from a batch program and the program is modernized to play its real-time role in an EDA.
Another way is to slice up batch files to messages and to queue up messages to batch-files; both at the perimeter of the ESB.
This is the way I explain the EDA-mindset to our analysts and programmers. And it works... (as far is they are old enough to know anything about batch programming ;-)).
Hi mike, previous message was mine...
We have a VSE mainframe among other things. Is that old school enough for you ?
;)
I like your analogy - I'm going to have to use that with some of our mainframers.
Thanks Jack!
Post a Comment