Algorithmic Trading Imperative #8: Design for Low Latency Decisions

In algorithmic trading, milliseconds matter. Minimizing the time between event detection (market data, news, requests for quotes) and action (placing an order) is critical. To do this, firms are using complex event processing (CEP) technology to implement their white-box algorithmic trading platforms. CEP is a new paradigm that allows organizations to identify patterns among streaming event data and respond to those patterns in microseconds (read a detailed overview of CEP and the history of its development here). Using a traditional database, you must store, index and retrieve the data – a very time-consuming process. CEP allows you to establish rules, or trading strategies, and ‘stream’ data through them, so the relevant data may be selected. This makes it possible to monitor, analyze and act on market data and respond immediately.

Many CEP engines were designed with unique in-memory architectures that ensure the continuous processing of events that can arrive in volumes of tens of thousands of events a second, process millions of concurrent CEP "rules," and make decisions in less than a millisecond.   The Apama "Correlator," depicted here, includes a patented technique of processing events based on an in-memory multi-dimensional indexing scheme called the HyperTree, in combination with a complex event sequencer, which optimizes Correlator performance by optimizing the processing of event scenarios that express the temporal and sequential event patterns that are expressed in CEP rules.  These rules can be expressed by using an eclipse-based development environment for the Apama CEP language or a high-level, graphical CEP Scenario Modeler that allows non-programmers (e.g., heads of desks) to "paint" strategies quickly and easily without programming expertise.

CEP correlation engines help fulfill imperative #8, to design algorithmic trading architectures for low-latency.