The instance design pattern in the Apama EPL

In this my third installment on a series devoted to the Apama Event Processing Language (EPL), I'll continue where I left off last time in which I described the basic design of event passing for a consumer/producer model.  For this revision I've extended the model to support multiple consumers.  This introduces the instance management feature of the Apama EPL.  Instances or 'sub-monitors' as they're often referred to define a discrete unit of work. The unit of work can be very small (an analytic calculation) or very large (a whole trading algo). Each instance gets spawned with it's own parameter set, listens for it's own event streams and operates in a very singleton manner. To which I mean, within the semantics of the application an instance need only be concerned about managing itself not other instances. Overall, it is a  factory behavioral model extended to include an execution model.  This is a key aspect of the Apama EPL, making a common application requirement simple to implement, robust in design, and highly performant in the CEP model.

The Apama CEP engine manages the execution of these sub-monitors (also known as mThreads internally). In a typical production deployment, there would be 100's or 1000's of sub-monitors running. The spawn operator is the single statement in the language that accomplishes this unique feature. Spawn is basically a self-replicating scheme with certain behavioral rules. The main rule: the cloned instance does not get the active listeners (i.e. on all someEvent...) of the parent. It must establish it's own. Actually it's the perfect model for that factory-like behavior.  The clone does not want it's parents listeners, but would create it's own based on the parameters passed such as the symbol name in a trading algo or the subscriberId in our Producer example below. Speaking of our example ...

For the sake of brevity, I've just listed the extended Producer side of my consumer/producer example below. For the complete example, you can download it here.

The (extended) Item Producer.

package com.apamax.sample; monitor ItemService {         event ClearUserID {         integer id;     }         UnsubscribeFromItems u;              integer count := 0;     float price := 0.0;     listener l;         action onload {         // list of subscriber (user) identifiers         sequence<integer> ids := [];                 SubscribeToItems s;         on all SubscribeToItems():s {             if ids.indexOf(s.subscriberId) = -1 then {                 ids.append(s.subscriberId);                             spawn startSubscriptions(s.subscriberId, s.item_name);                   }         }                 ClearUserID c;         on all ClearUserID():c {             log "in " + c.toString();                integer index := ids.indexOf(u.subscriberId);                 if index != -1 then {                     ids.remove(index);                 }         }     }     action startSubscriptions(integer this_subscriberId, string name) {         log "in startSubscriptions";                 l := on all wait(0.1) {             route Item(this_subscriberId, name, count, price);             count := count + 1;             price := price + 0.1;         }         on UnsubscribeFromItems(subscriberId = this_subscriberId):u {             stopSubscriptions(u.subscriberId, u.item_name);         }            }         action stopSubscriptions(integer subscriberId, string name) {         // do something to stop routing events         log "in stopSubscriptions";         l.quit();         route ClearUserID(subscriberId);     } }

To get a general sense of what this bit of code is intended to do, I suggest a quick scan of my previous installment where I introduced this example.

The extended Item Producer is expected to manage multiple uniquely identified consumers. For that it must maintain a list of identifiers, one for each consumer.  It does that by appending and removing entries from an array (sequence<integer> ids).  his is a common idiom for tracking identifiers, syntactically it's similar in many imperative languages.

This example uses a single-cast event passing scheme where the Producer routes Item events uniquely tagged to the consumer (route Item(this_subscriberId, name, count, price)).

 On the consumer side, Item events are listened for based on a subscriberId (on all Item(subscriberId = myId)). It's the uniqueness of subscriberId (one per consumer) that defines this as a single-cast design. A common twist to this a multi-cast event passed scheme (not be be confused with the UDP multicast) where multiple consumers might be requesting the same information (i.e. the item_name in our example).  A well understood example of this would be a market data adapter providing trade data for the same symbol to multiple consumers.  The Item Producer would change very little to support a multi-cast event passing scheme.

In the listener "on all SubscribeToItems()", we spawn to the action startSubscriptions when we receive a  SubscribeToItems event from a consumer. We pass the parameters of  the consumer's identifier (s.subscriberId) and the item (s.item_name) to instantiate the new sub-monitor.  A new mThread of execution is created for the sub-monitor and it begins executing producing Item events. The parent monitor continues waiting (listening) for another SubscribeToItems event.

 You'll also notice the use of a private event ClearUserID, the purpose behind this is to communicate between the spawned sub-monitor(s) and main (parent) Monitor when an UnsubscribeFromItems request is received.  This is necesssary since the parent monitor manages the id's of connected consumers. A spawned sub-monitor uses this event to simply inform of termination.

The event paradigm in the Apama EPL extends far beyond the notion of processing data events. In one sense you could categorized events as data and control. Data events are consumed and processed by the CEP application. Control events direct the semantics of the application. 

 

This example is designed to illustrate a few powerful yet easy to use features of the Apama EPL:

1. To highlight that the notion that managing multiple consumers (clients) becomes a simple and safe programming task in the Apama EPL. Instance management is an intrinsic design pattern based on the commonly understood factory model. We did not reinvent the wheel, we simply refined it and made it approachable in the CEP paradigm.
2. Events are not just application data to be processes by monitors. They provide semantic control of an application as well.

Here's the complete example with the consumers, interface and producer. 

Once again thanks for reading,
Louie