Get Yourself Connected
Posted by Richard Bentley
Our integration with Vhayu Velocity, announced earlier this week, is the latest connectivity option we've added to the Apama platform. This follows hot on the heels of the launch of our adapter for the Brazilian BOVESPA market, and got me musing about the importance of connectivity for our Apama CEP platform - and the effort we need to invest to develop and maintain it. All in all we now support more than 40 distinct adapters (as listed here) and are adding more all the time (we have 5 new adapters in development right now).
The importance of connectivity cannot be overstated. One could have the best CEP Engine on the planet, but if there's no way of getting events in and out of it then it's of zero use - the proverbial Ferrari with no wheels. For basic infrastructure, you can go some way with strong support for standard middlewares through JMS, databases through ODBC/JDBC, etc. For Capital Markets however, where latency is often a critical success factor, it is often about direct connectivity to the market, involving development to proprietary and extensive market-specific APIs; although good support for the FIX protocol is most definitely necessary, it is nowhere near sufficient for the low latency high frequency trading world.
Of course, once you have strong connectivity to a particular market then that becomes an enabler - our support for the native market data and order execution APIs of the Chicago Mercantile Exchange (CME), for example, is allowing Apama to play in the proprietary trading world of the Futures and Commodities markets with great success, and it is no surprise that Apama is doing particularly well in Brazil! But such connectivity does not come cheap; as exchanges regularly rev their APIs, adapters need constant care and feeding, and there are always new markets and systems that need to be connected, particularly for a platform like Apama which plays across all asset classes, on a global stage. Apama currently retains a team of 10 Engineers and QA staff who *just* build and maintain our adapters, and we regularly second other personnel to this team to cope with demand spikes.
We took a decision early on to invest in developing our own connectivity; we clearly had options - there are no shortage of intermediaries out there who purport to connect to 000s of different systems. But there's always that one system that they don't support - or don't happen to support on the platform you need - so you never get away from having to build and maintain it yourself to some extent. And then of course there's the need to integrate with proprietary systems - when you start off by targeting your product at the Tier 1 investment banks, there's an awful lot of in house systems you need to hook up to!
In fact, our earliest clients were all of this kind, requiring proprietary connectivity to their home-grown systems. Overall this has been a boon for Apama, as we were forced to focus almost day 1 on building a toolkit which allowed us to rapidly develop new connectivity – resulting in our Integration Adapter Framework (IAF). All Apama connectivity is built using IAF - and as a result benefits from common configuration and management interfaces, uniform latency measurement framework, real-time status reporting and more.
Building and supporting the IAF, developing our 40+ (and counting) adapters, keeping up with exchange upgrades and so on requires a huge investment of time and effort. Whilst this might be a much less glamorous aspect of developing a CEP Platform (my colleagues in Engineering touchingly refer to it as "the filth"), it is a hugely critical one.
... and whatever happened to the Stereo MCs anyway?