Event Processing blog

Apama 4.2 release - Cruising in the fast lane

The Apama engineering team has done it once again. True to our record of releasing significant new features in the Apama product every 6 months, the v4.2 release is hot off the presses with major new functionality. The Apama roadmap is driven by a keen sense of our customer requirements, the competitive landscape and an opportunistic zeal. The engineering team is a dedicated R&D team driven to excellence and quality. We are dedicated to delivering value to our customers. A consistent comment we've heard from analysts and customers alike is the maturity of the Apama product.  

The current v4.2 release, the third in the v4.x family adds significant enhancements in three concurrent themes - Performance, Productivity and Integration. This consistent thematic model is one we've held for a number of years. Below I've touched upon the highlights of the current release along these themes:

• Performance

High Performance Parallelism for Developers.  The Apama Event Processing Language (EPL) provides a set of features uniquely suited to build scalable event-driven applications.  The language natively offers capabilities for event handling, correlating event streams, pattern matching and defining temporal logic, etc. Equally important, the language provides a flexible means to process events in parallel.  For this we provide a context model and a new high performance scheduler. Contexts can be thought of as silos of execution, where CEP applications run in parallel. The scheduler's role is to manage the runtime execution in an intelligent high-performance way, and to leverage the underlying operating system threading model. It’s via the context architecture that the Apama Correlator squeezes the most out of operating system threads to achieve maximum use of multi-core processors for massive vertical scalability. For IT developers, this is a effective and efficient means to build high performance, low latency CEP applications without the pitfalls of thread-based programming, such as deadlocks and race conditions.

High Performance Parallelism for Business Analysts.  Not to be left out of the race, we've also ensured the scalable parallelism provided in the Apama CEP engine is available through our graphical modeling tool, the Event Modeler. We've had this graphical modeling capability since the very first release of Apama. This tool designed for analysts, quantitative researchers and of course developers, allows you to design and build complete CEP applications is a graphical model.  Parallelism is as easy as an automatic transmission, simply select P for parallel.

• Productivity

Real men do use Debuggers (and Profilers too). The Apama Studio now sports major new functionality for development, a source level debugger and a production profiler. Building applications for an event-driven world presents new programming challenges. Having state-of-the-art development tools for this paradigm is a mandate. The Apama EPL is the right language for building event-driven applications - now we have a source-level debugger designed for this event paradigm. Available in the Eclipse-based Apama Studio it provides breakpoints to suspend applications at specific points, examine contents of program variables and single stepping. It works in concert with our parallelism as well. Profiling is a means to examine deployed Apama applications to identify possible bottlenecks in CPU usage.

Jamming with Java. We've enhanced our support for Java for building CEP applications. The Apama Studio includes a complete set of wizards for creating monitors, listeners, and events to improve the development process when building java-based CEP applications in Apama.

• Integration

The (relational) world plays the event game. While we have provided connectivity to relational databases for many years we've made a significant re-design in the architecture of how we do it with the new Apama Database Connector (ADBC). The ADBC provides a universal interface to any database and includes standard connectors to ODBC and JDBC.  Through the ADBC, Apama applications can store and retrieve data in standard database formats using general database queries, effectively turning these relational engines into timeseries databases. The data can be used for application enrichment and playback purposes. To manage playback the Apama Studio includes a new Data Player that enables back-testing and event playback from a range of data sources via the ADBC. One can replay at varying speeds event data and time itself. The tested CEP applications behaves temporally consistent even as data is replayed at lightening speed.

Cruising at memory speed with MemoryStore. The MemoryStore is a massively scalable in-memory caching facility with in-built navigation,  persistence and visualization functionality.  This allows CEP applications, which typically scan, correlate and discard data very quickly to retain selected portions in memory for later access at extreme speed. This could be for managing a financial Order Book, Payments or other data elements that the application needs to be able to access at user’s requests quickly. Furthermore, if required the in-memory image can be persisted to a relational database for recovery or other retrieval purposes, and lastly the MemoryStore allows selected portions of the in-memory cache to be automatically mapped to dashboards.

Well that's the highlights. There were also about a dozen other features within each of these three themes, just too numerous to mention.

We are committed to improving the Apama product by listening to our many customers, paying close attention to the ever-changing competitive landscape and researching new opportunities.

Again thanks for reading, you can also follow me at twitter, here.
Louie

If You Build It They Will Come, An Apama Algo Webinar

My colleague Dan Hubscher and I just finished the first of a two part Webinar entitled "Build Quickly, Run Fast". In this Webinar we explained and demonstrated Apama as an Algo platform for high frequency and order execution algorithms.

As I've blogged in the recent past it is an arms race in High Frequency trading.  The need to build quickly is a demanding requirement to keep ahead in the race. Being armed with the right tools is paramount. Rapid development and customization of strategies using graphical modeling tools provides the leverage necessary to keep pace with fast moving markets.

To that point, in this webinar I demonstrated a couple of algo examples. The first was a complete strategy that incorporates an alpha element with multiple order execution options. In  designing and building strategies the trading signal detection is just the first part of the problem. This typically involves an analytic calculation over the incoming market data within some segment or window of time. For example a moving average calculation smooths out the peaks and valleys or the volatility of an instrument's price. Once the signal is detected it's time to trade and manage the order's executions. This is a key distinction between other CEP products and the Apama platform for building trading strategies. While it's possible to define an Event Flow in most or all CEP products for data enrichment and data analysis (i.e. the signal detection), for most other CEP products you have to switch out to some other environment & language to build the rules to manage the executions. The Apama platform is about building complete event-driven applications. So trade signal detection and order executions, whether it's a simple iceberg execution or something much more complex it can easily be designed, built and backtested in the same Apama graphical modeling environment (Of course for those more inclined to traditional development tools and methodologies, Apama offers a full suite of developer tools, an EPL, debugger, profiler and java support).




The second example in the Webinar demonstration was to build a small, but working strategy from scratch. I did this live in full view of the attendees. For this I did a basic price momentum strategy. This tracked the velocity of price movements. The trading signal was a parameterized threshold which indicated when that price moved up (or down) a specific amount for a specific duration.

This webinar is focused on highlighting the ever-present challenges investment firms face in high frequency trading:

• Fears of the Black Box
• The simple fact that markets are continually evolving
• First Mover Advantage
• Customization is king

Along with my colleague Dan Hubscher,  the Build Quickly webinar describes how the Apama platform delivers solutions to the Capital Markets industry to meeting these needs and challenges. 

Stay tuned for a link to the recording and don't forget to dial in to part II where we focus on performance requirements and characteristics. Again thanks for reading (plus watching the webinar), you can also follow me at twitter, here.

A follow up note, here's the link to the recordings for both part I and part II on Build Quickly Run Fast.

Louie

Progress Apama Capital Markets in Brazil

In recent years, the Brazilian market has grown stronger, and become very aggressive with algorithmic trading. Just back from a conference in Brazil, listen to this podcast where Dan Hubscher shares insight into the current state of Brazil’s market, and what the people down there are buzzing about.

Did You Know?

What a busy summer!  Here is a brief update on what’s happening with Apama in Capital Markets lately.

In the News

High frequency trading is in the news.  Can you handle THIS perfect storm?

Events

In Chicago with CQG on August 6th:  Join us HERE to see a live demonstration of the integration of CQG’s trade routing and technical analysis tools with Progress Apama’s market-leading platform for algorithmic trading.

In Brazil with Thomson Reuters in August:  Join us in São Paulo on August 11 and in Rio de Janeiro on August 13 for lively discussions on quantitative trading.  (Need a translation from Portuguese?  Try Google Translate – Sao Paulo/Rio de Janeiro)

Awards

1 MONTH LEFT TO VOTE!  You have the chance to decide what you consider to be the best systems in the Banking Technology Readers' Choice Awards 2009. 

Apama wins Best Algorithmic Trading Platform from Profit & Loss Magazine’s 2009 Digital Markets Awards for second consecutive year (Press Release, Blog).

Connectivity Update

Did you know that Apama is certified to connect seamlessly with the Lime Trading System?

Webinars

Replay recent Capital Markets webinars from Apama:

• Real-Time Risk – A Mandate for Trading in Volatile Markets
• Apama Algo Trading Accelerator: Build Quickly, Run Fast
• FX Aggregation and Beyond: Exploiting the Power of CEP
• Energy Trading And Real-Time Risk Management - Two Sides Of The Same Coin?
• CEP – Strategic or Tactical Weapon

Some are calling current market conditions "a perfect storm," but should you lower your sails and wait for calmer conditions? There are other choices available to you – and smart firms are finding ways to leverage those choices. 

Until next time, keep trading!

-Dan

Apama SIFMA 09 Announcements

SIFMA is happening this week in NY and this typically sparks a flurry of announcements in Capital Markets, as this event can be seen as a benchmark for what is happening in the industry, similar to TradeTech Paris for Europe earlier in the Spring.  Apama has made a series of significant announcements that capture some of the scope of the Apama platform and what we believe is required to be successful in this market.  In different niches of the blogosphere you’ll find some proponents of fairly narrow definitions of what “CEP” is or how one measures CEP “leadership” or “maturity”.  I think the Apama announcements illustrate a different perspective, reflective of the breadth of the Apama platform and the possibilities available to such a platform in terms of building new event-driven solutions in Capital Markets.

Market Surveillance and Monitoring Accelerator

Apama has previously announced market surveillance customers, FSA and Turquoise, and with our new enhanced Solution Accelerator we are capturing our capabilities in a way that allows regulators, exchanges/MTFs, and trading firms to further jumpstart deployment of monitoring applications.  The idea that it would be efficacious to have real-time monitoring of trading is becoming better understood in the indusry.  If you read the release, you’ll note that the targets are not just regulators or exchanges, but firms themselves, who would benefit from detecting potentially damaging activities prior to that activity hurting firm reputations or profits.

Apama Solution Accelerators provide a set of core functions that focus on specific domain areas, but they still retain the flexibility to evolve and adjust as circumstances require.  That is key to keeping pace with very dynamic market conditions.  These Accelerators have proven a key driver to our recent success as they marry the power of the underlying platform with the real “end game” of providing customers with solutions that deliver value.

UniCredit Customer Win

As an example of the power of Solution Accelerators, Apama also announced that UniCredit is using the Apama FX Market Aggregator (another of our roster of Accelerators) to give their FX traders access to prices from a number of FX liquidity venues.  UniCredit is also using the Accelerator to publish FX prices to its eFX downstream channels.   Again, a key point here is that the use of an Accelerator in no ways constrains a client from using the Apama tools that are part of the underlying platform to build out other capabilities that complement the Accelerator.  For some the Accelerator is close to what is needed, but for others it is an attractive launching point.  For further discussion of this, you might want to check out a recent Webinar on “FX Aggregation and Beyond” that talks to this issue.

Lime Brokerage

We added to our broad range of connectivity adapters with  an important connection to Lime Trading System’s Citrius market data feed and FIX order placement services.  This give our customers access to Lime’s trade execution capabilities for equities, derivatives, ETFs, futures and options.  Apama customers can also look to Apama applications via Lime Trading System’s collocation facilities.  Lime is a really cool firm and we see this as a great opportunity for our common customers – both those now and to come.

Connectivity is the lifeblood of CEP and other event-driven applications.  Apama has a broad range of adapters, and we have an engineering team whose specific focus is on this aspect of the product platform.  Integration adapters aren’t “sexy” and they don’t get a lot of attention in marketing literature, but they are vital and deserve a bit of spotlight.

BondDesk

BondDesk provides 2,000 broker-dealers with access to 35,000 live and executable offerings from 120 premier fixed income dealers.  In an announcement this week, we announced that BondDesk ATS (alternative trading system) customers will be able to register their interest in the availability of fixed income securities that meet certain criteria and Apama will monitor the inbound data and provide real-time notification when a matching offering becomes available on the ATS.

So four announcements this week, and more upcoming.

So stay tuned. 

.

High Frequency Trading driving the need to build quickly, run fast

In just about any race there is usually a starting point and a finish line, unless of course you are in an arms race.  For that sort of race there may have been some nebulous beginning in the distant past, but there is no finish line. The race just keeps sprinting along, each competitor angling for an edge, regularly recharging their ammunition supply with some new weaponry to get ahead however slight or temporary.

I recently read an interesting article describing High Frequency Trading as embroiled in an arms race. I certainly believe it's well entrenched in such a conflict, but frankly this combat has arguably had a beneficial net effect especially in that it's contributed to the wellspring of invention, inspiring the creative spirit in all the supporting attributes that make High Frequency Trading a reality. Behind any trader (and trading firm) is an entire armada including the vendors supplying the underlying hardware, networks, software platforms and trading applications.  They are all immersed in the war.  As new hardware, software and/or algo's are deployed it allows the trader to do battle and speed ahead even if it's just for a short while.  Competitive pressures, increasing market volatility, regulatory imperatives, risk mitigation and a host of other challenges are the land mines and roadside bombs on the long and winding road that stall and slow causing re-tooling and re-stocking the ammunition (i.e. algo strategies). There is no time to stop and catch your breath or stand on the roadside.

Sang Lee from the Aite Group reports that High Frequency Trading has had a significant impact on the overall market, providing greater liquidity, tighter spreads and overall improving the quality of the market.  At the macro level these are great advancements and mark a natural evolutionary step due to so many market changes in recent years (i.e. electronic trading venues, adoption of CEP platforms for algo trading, etc.) in Equities and beyond (i.e. FX and Futures & Options).  Down in the trenches, the battles rage on day by day as a multitude of traders and an untold number of algo strategies provide the market liquidity by moving in and out of positions in milliseconds (or even less time). The trading firms engaged in this never ending conflict drive a set of imperatives on software infrastructures for building and deploying algos in the High Frequency battlefield:

Rapid development and customization of algo's

Algo strategies in the High Frequency world have a limited life time. They soon become obsolete (i.e. whatever alpha they took advantage of has disappeared due to the competition, economic changes, or other situations).  To react and respond to this inevitability, having the right sort of tooling to recalibrate strategies is a necessity. This includes graphical modeling tools for Quants to prototype ideas quickly, backtest with historic data, test in a scalable manner to instill confidence prior to production rollout and lastly dynamic parameterization of strategies from graphical dashboards. Not forgetting the code-slinging types, an Integrated Development Environment (IDE) for support of event processing language (EPL) development for more low-level tasks.

Abstracting over increasingly complex strategy logic

Supporting Quants with a rich and robust set of functionality from the basics (connectivity to markets) to the advanced (Linear Algebra, Black Scholes, and other statistical functions).

Support for the 'ilities (availability, security, reliability, ...) to manage the mundane

Deploy with confidence. An important role of software infrastructure is to instill confidence that deployed strategies are always available, securely accessed and run without failure.

Support for scalable performance, providing high throughput and low latency

This is probably the most paramount requirement in the arms race of High Frequency Trading.  The race to the microsecond is pushing both hardware and software vendors alike. Parallelism in CEP engines like Apama's Correlator can leverage multi-core processor architectures like the Intel Nehalem. 

Along with my colleague Dan Hubscher,  I have recorded a 30 minute webinar that describes how the Apama platform along with the Apama Algorithmic Trading Accelerator meet these imperatives.

The pre-recorded webinar, is available here:  Apama Algorithmic Trading Accelerator, Build Quickly, Run Fast.

Once again thanks for reading (plus watching and listening to the webinar in this case), you can also follow me at twitter, here.
Louie

Apama Wins Profit and Loss FX Award for Second Year in a Row

Apama has received Profit & Loss Magazine’s 2009 Digital Markets Award as Best Algorithmic Trading Platform.  This is the second consecutive year that Apama has won this award from one of the FX industry’s leading publications.  The Digital Markets Awards recognize the efforts of the FX services industry in providing the tools and functionality that make trading FX more efficient.  The presentations of the awards to the winners in each category were made at the Profit & Loss Forex Network New York 2009 Event Awards Dinner.

The award winners in each category were determined by your votes.  As with any awards like these, your voices are heard loud and clear.  So thank you, to all of you that voted.  And remember, you can make your voices heard again, by voting in the Waters Rankings for 2009.

-Dan

We're going on Twitter

Louis Lovas and myself, Giles Nelson, have started using Twitter to comment and respond to exciting things happening in the world of CEP (and perhaps beyond occasionally!).

The intent is to complement this blog. We'll be using Twitter to, perhaps, more impulsively report our thinking. We see Twitter as another good way to communicate thoughts and ideas.

We would be delighted if you chose to follow our "twitterings" (to use the lingo), and we'll be happy to follow you too.

Click here to follow Louis and here to follow Giles (you'll need to signup for a Twitter account).

The World According to a 4-Year-Old, and the Real Business of Foreign Exchange

The following conversation with my 4-year-old daughter happened just a few days ago:

Her:    I really like China, it sounds like a great place.

Me:     Why?

Her:    They have the best toys in the world there.

Me:     How do you know?

Her:    All my favorite toys are "Made in China."

I stopped asking her questions.  But my mind quickly went from wondering about the origins of toys to wondering about the origins of the capital markets business applications I encounter these days.  These applications rely heavily on functions like alpha-seeking and execution algorithms, and foreign exchange aggregation, to name a few.  I’ll elaborate on others in future posts, but the point is that this is a high risk environment, not child’s play.  So where do the ideas for those functions come from? 

A few Apama customers I have met lately note that the markets seem to be acting like a bunch of 4-year-olds having temper tantrums.  They noted wild swings in the FX marketplace, for instance, since the credit crunch became a crisis a few months ago.  FX trading will continue to be frantic this year. But necessity is indeed the mother of invention, which is a good thing with the markets the way they are.

This week, we announced that Standard Chartered Bank has gone live with the Progress Apama FX Market Aggregation Accelerator.  We’ve talked about FX aggregation before, and have done a number of similar Tier 1 deployments.  As you would expect, some great ideas in these projects have been customer-driven.  And when I look behind the scenes at Apama, I see how these projects provide a springboard for people to think of new things and try them out.  

On my very first day at Apama, a new colleague walked up to me and said, “I solved a problem that was bugging me.  Do you think customers might like this too?”  These ideas tend to grow into a demo, which may become the basis for a collaborative deployment, and eventually a customer has built on the idea and taken it in a new direction – a similar Apama story can be found here. 

Yesterday someone asked me if I think that Capital Markets firms are using technology to save cost or to grow.  Cost savings?  Yes, at least, and for some that may be the end of the story and the mind is closed.  But a former manager of mine, now a colleague and adviser, taught me to listen to the markets by telling me, “You have 2 ears and 1 mouth, use them in that proportion.”   A similar philosophy serves software firms, as well.

Funny that this week, one of my Wall St. friends at a very alive firm wrote to tell me his opinions about the state of software evolution, and the importance of innovation.  I think the crisis will force patches of growth and innovation.  Do you?  Comments welcome.

 

-Dan