In Complex Event Processing, is “Un Oeuf Enough?”

(co-written by contributor Dr. Bart Schouw, Business Development Manager, EMEA)

Why do French recipes commonly prescribe to use only one egg in a recipe?  Because “un oeuf is enough.”  The same applies for event driven applications.  While many complex event processing (CEP) vendors harp on the ability to process 100’s of thousands of events per second (EPS),  the main value of CEP lies in an effective approach to event driven application architecture, not just the ability to handle large volumes.  It’s true that there are high-end CEP applications out there: in the financial services world of options pricing, the OPRA (Options Price Reporting Authority) penny-price feed will deliver 716,000 events per second (EPS) by January 8, 2008, and some of the CEP engines, like Apama, are built to handle these kinds of event rates.  But such EPS rates are the exception, rather than the rule, and the real value of CEP is much deeper than processing speed.

Outside of capital markets, the EPS demands for important event driven applications are often more modest.  For example, an Apama retail customer, in a relatively large automated warehouse management, has 75 pickers picking 500 items an hour, yielding about 40,000 order “events” an hour and 40,000 response “events” an hour (“done” or “unable to pick”).  That’s a total of 80,000 events an hour, or about 22 events a second.

22 events a second can realistically be handled by a traditional RDBMS solution, if the only technical challenge was about handling event volume. The leading CEP platform allow event-driven business logic to be expressed with a concise, powerful language that requires a fraction of the code required by traditional, static programming techniques. CEP languages are built on event-driven constructs (a “WHEN (event) >> THEN (logic)” metaphor), concisely identify event sequences (A followed by B, then C), and natively understand temporal constraints (with 2 hours).   So CEP concisely models the event-driven world – something static computing metaphors can’t easily do.

So back to our supply chain example, if real-time traffic and routing status and picker actions can be correlated, CEP rules can intelligently optimize picking to load a truck that’s at risk of on time and therefore must leave sooner because of traffic congestion and expected delivery times.  This customer wanted to express a complex correlation from its retail operation in Istanbul.  The logistics challenge was to replenish their outlets by automating a new centralized warehouse that replaced six small warehouses.  The new, massive warehouse they built was far from the center of Istanbul, which was far away from the most profitable shops were on the European side of Istanbul.  Out-of-stock situations at the European side would have a big impact on the profitability of the retailer. To make matters worse, the only way to get from the Asian side to the European side was via two bridges, which frequently had traffic jams.  To add to the complexity, in order to ease traffic burden, the government ordered that trucks were not allowed after 7:00 AM, making it crucial to get trucks over the bridge at the right time.  Previously, the picker’s plans were printed and handed to pickers, and couldn’t be changed. 

Correlating stock, picker schedules, traffic congestion, and truck availability helps automate the picking process with hand-held devices that dynamically displayed the next pick on the floor.  A central CEP application provides the manager visibility into the picking process to see if, for example, a delay was imminent due to traffic jams at the bridge, and gives him the real-time ability to optimize pick lists by picker on-line, re-directing resources. Extensions of this approach allow new CEP scenarios to rebalance operations intelligently and predictively, based on history and real-time inputs such as weather and traffic.

The value event-driven applications like these isn’t the ability to process hundreds of thousands of events a second, it is to express event-driven business logic quickly and easily (read on CEP and empowering business users), and to evolve event rules quickly as conditions change.  Some CEP tools allows business users to participate in this process as well, enabling domain experts that, in this example, understand supply chain. 

So CEP can quickly solve event-driven business problems, and provides value far beyond the ability to crunch numbers.   So one egg, the egg of performance, is often a key ingredient, but is not fully sufficient to create a full meal.  To provide complete business value, a comprehensive approach to event processing is required to rapidly compose, deploy, and evolve event driven logic.

Comments

[Brian Connell, CTO WestGlobal]

A small typo - 80,000 events per hour is 1,300 events per minute, not per second (approx 22 per second).

[Mark Palmer]

Small typo, but a big difference! Thanks, Brian, I fixed this in the post.