Current Affairs

Tuesday, November 10, 2009

Event Processing in Location-based Services

Posted by David Olson

Business is event-driven. No. Wait. Life is event-driven, and if it wasn’t, we’d be walking into walls and every sentence would start with “Oops.” Life would be a string of missed opportunities. We’ve done a masterful job of using technology to transform our business processes into software, but one tenet that’s been missing is that business should imitate life. Sense and respond is what’s been missing.

We recently announced that match2blue (http://web.progress.com/inthenews/match2blue-stands-ou-11092009.html) will be using the event processing capabilities of Apama to provide location-based services in social networking. Sense and respond is crucial for their ability to enable like-minded people to connect in real-time. Traditional data processing technology and its normal rhythm of “capture, store, analyze” can’t, well, keep up. And in a world where latency leads to missed opportunities, match2blue is proving that through the right technology business can imitate life.

Responding to business events as they happen is what will define your competitive advantage.

Business is event-driven, indeed.

Thursday, November 05, 2009

In defence of high frequency trading

Posted by Giles Nelson

The high frequency trading (HFT) debate seems to have entered a new and worrying phase in the UK. On Tuesday this week in an interview with the BBC, Lord Myners, the UK’s financial services minister, warned that high frequency trading had “gone too far” and that share ownership had “now lost its supporting function for the provision of capital to business”. (You can find the original interview here and reports of it in the Financial Times and The Independent yesterday).

 

 Mary Schapiro, head of the SEC, signalled at the end of October that a number of electronic trading areas were going to be looked into – naked access (where a broker sponsors a firm to have direct electronic access to an exchange), dark pools and high frequency trading.

 

It does seem now that on both sides of the Atlantic, governments and regulators are steeling themselves to act and softening the markets up to be able to accept the fact that electronic trading might have some limits.

 

The concern is that governments and regulators are going to come down too hard on electronic trading and the benefits that it gives investors will be damaged.

 

It all started with the flash order issue in the US a few months ago. Commentators were linking together various different, although related issues, in an inappropriate way. Flash orders seemed to be viewed sometimes as being synonymous with HFT, both of which were sometimes reported as forms of market abuse. All three topics are quite different. In my opinion, there are legitimate questions over the use of flash orders and a proposal to ban them is now being considered.

 

Dark pools, where large blocks of stock are traded off exchange to minimise market impact, have been the next targets. There are, again, legitimate issues. Dark pools, by their very nature, do not have good price transparency. Regulators have become concerned with their use because more and more trading is going through dark pools. Some estimates put this at between 10% and 30% in Europe and the US. This lack of knowledge about what exactly is the proportion is part of the problem itself. No one really knows what proportion of trading dark pools is taking. If a significant proportion of the market has no price transparency then this undermines the notion of a fair market for all. Regulators are looking at this and its likely that they will force dark pool operators to disclose far more information about what is being traded than they do currently. The SEC is considering limiting the proportion of a stock that can be traded through dark pools to a small percentage.

 

These legitimate issues however risk skewing the whole HFT debate to one where people will conclude that “HFT is bad”.

 

What people are now describing as HFT – the very fast and frequent, computer assisted trading of, usually, equities – is an evolution of something that has been happening in the market place for at least the last 10 years. In this time electronic trading has proliferated, not just in equities but also in all asset classes such as derivatives, bonds and foreign exchange. Far more venues for trading have been created. There are now many places where a company’s stock can be traded both in the US and Europe. This has brought competition and choice. Prices have been lowered, improving access to retail investors. Spreads have narrowed. Arbitrage opportunities are harder to find, which mean that market information is disseminating faster which, in turn, means that price transparency has improved. Because there is more trading going on, there is more liquidity available, which also means keener prices.

 

A key part of the HFT trend has been the use of algorithmic trading (the most prevalent use of complex event processing technology). Algo trading models fall broadly into one of two camps: alpha seeking, where market prices are examined to find a trading opportunity that will make money, and execution where orders are, usually, split up into smaller parts and then traded automatically in the market in an intelligent way to find good prices and to ensure those prices are not overly influenced by the trades being made themselves. For each type of model it can be very useful to react very quickly to market information, either to take advantage of a price discrepancy or to quickly pickup liquidity at a good price. Algorithmic trading is enormously beneficial for those who use it and its use is not limited to specialist hedge funds. Most algorithmic trading uses execution models that find liquidity, good prices, help minimise market impact and, lastly, increase significantly a trader’s productivity. Instead of wasting time executing several simple orders in the market over the course of many minutes or hours, the trader can simply ask a machine to do it. The trader can then spend time either covering more of the market (useful in straitened economic times) or spend more time actually delivering real value to a client.

 

Algorithmic trading and HFT have brought very significant benefits. It is these benefits that must not be threatened.

 

Trading has always involved cunning and guile, whether human or computer based. Competition has always existed in who’s got the best traders and trading systems. Organisations investing in ultra low-latency infrastructure to ensure orders arrive at an exchange in microseconds (not nanoseconds as sometimes claimed by the way – light travels 30cm in 1 nanosecond which isn’t far enough to be very useful) are part of this competitive world. Competition leads to innovation and it is this innovation that has brought so many of the benefits described above. Computer-based models can somtimes be used abusively. There are many forms of market abuse that regulators and exchange operators look for. Some exchanges and regulators have been investing in real-time surveillance technology (Progress counts Turquoise and the UK Financial Services Authority as customers using Apama) to ensure that they can spot abusive patterns of behaviour quickly.

 

We can’t start slowing trading down. We can’t go backwards and put the electronic trading genie back in the bottle. We don’t want to lose all the benefits that have come. Rather, regulators and exchanges should concentrate on ensuring maximum transparency in how markets operate and ensure that those attempting to maliciously abuse the markets are dissuaded or caught.

 

Sunday, August 09, 2009

Riding the Crest of the Wave... the Forrester Wave

Posted by Louis Lovas


In just a few short days of its announcement news of the Forrester CEP Wave has spread to all corners of the globe. From trade magazines, online journals and blogs to Facebook and Twitter, the headlines are everywhere. A Google search yields thousands of hits.  "Independent Research names Progress® Apama® as a Standout Leader in CEP ..."

The Forrester team of Mike Gualtieri and John Rymer state 'The Fledgling CEP Platform Market Is Vibrant, Competitive, And Dynamic'. Of course those of us that have been immersed in event processing for the past few years already knew that. It was our job to convince Mike and John. On behalf of Progress Apama and the CEP community, I would like to extend a word of thanks and appreciation to both of them for their efforts, diligence and patience in putting this Wave together. An enormous task given they reviewed 9 CEP products and vendor strategies in depth. Considering this was the first CEP Wave they also had to define an initial blueprint on CEP by which to evaluate vendors, they did a commendable job. Well done. You can get a complimentary copy of the pdf version from us here.  

It was quite a few months ago when I and a few of my esteemed colleagues began the CEP Wave process. In the abstract it was not too much different from responding to the questions in a prospect's RFP/RFI, for which I and my colleagues have much practice. However, a difference that I found unique was the format. A client proposal is generally a Word document where one can provide plenty of written detail, and diagrams to depict product architecture and function. Forrester Waves are MS Excel spreadsheets. Vendor's responses to the Wave's questions are to fit into an Excel cell. Being a long-winded person, it was a challenge to have the necessary succinctness dictated by the confines of a cell.  My colleagues were quite helpful to this end. 

In short order, it became clear as to the benefits of the spreadsheet format. While many documents - proposals, reviews, evaluations or other become static paper the moment they're published that is not the case with Forrester's Wave. There is a clear intent behind Forrester's use of the spreadsheet format; it creates a living/dynamic document for their clients.  Spreadsheet's by their very nature can be interactive. Spreadsheet formula's can accept user input and recalculate. This capability is exactly what Forrester leverages in the CEP Wave.

The Forrester CEP Wave is divided into three categories:
  • Current Offering: A platform feature breakdown, development and deployment tools and performance characteristics.
  • Strategy: The vendors investment for the future.
  • Market Presence: Customer base.
Within each of these categories is an entire litany of subcategories containing features and criteria by which the product and vendor are measured. Each is assigned a weight as deemed appropriate by Forrester in reviewing the CEP industry at large.  Each vendor is then judged by their merits and scored. The most important aspect of this is the weighting. This is the key that gives the Wave that dynamic nature. From a client perspective, the weighting can be adjusted to suit your specific requirements. If for example, your shop is Windows-only you don't need to have a high weight on multi-platform support, you can lower that value. Likewise, if you have strong need for high availability/disaster recovery you can increase that weighting. Making these adjustments will tune the Wave for your specific requirements. You will then see how vendors stack up against each other with your customized weights. By doing so, what you will find is that the Apama platform pops to the top of the list all too often.

Once again thanks for reading, you can follow me at twitter, here.
Louie



Tuesday, August 04, 2009

Forrester Wave Cites Apama

Posted by Chris Martins

Forrester Research, a leading independent research firm, has just published its “Wave” on CEP and Progress Apama has been judged to stand out as a leader.  The evaluation process is quite detailed and lengthy.  It addresses:

·        Current product offering: product architecture, features, development environment, administration, interoperability, etc.

·         Vendor strategy: vendor’s product road map and other key strategy elements.

·         Market presence: market presence in terms of customer base, vendor size and presence, etc.

We are very excited to have Apama recognized in this way.  Together with the validation of our 115+ customer implementations (we don’t just score well in evaluations, but also in the real world of customers), this report speaks both to the vibrancy of the market and our leadership position.  But don’t take my word for it.  For a look at the actual report, including the graphical representation and the tabular scoring, you can check it out for yourself here.   

Sunday, June 21, 2009

High Frequency Trading driving the need to build quickly, run fast

Posted by Louis Lovas

<p>High Frequency Trading driving the need to build quickly, run fast</p>


In just about any race there is usually a starting point and a finish line, unless of course you are in an arms race.  For that sort of race there may have been some nebulous beginning in the distant past, but there is no finish line. The race just keeps sprinting along, each competitor angling for an edge, regularly recharging their ammunition supply with some new weaponry to get ahead however slight or temporary.

I recently read an interesting article describing High Frequency Trading as embroiled in an arms race. I certainly believe it's well entrenched in such a conflict, but frankly this combat has arguably had a beneficial net effect especially in that it's contributed to the wellspring of invention, inspiring the creative spirit in all the supporting attributes that make High Frequency Trading a reality. Behind any trader (and trading firm) is an entire armada including the vendors supplying the underlying hardware, networks, software platforms and trading applications.  They are all immersed in the war.  As new hardware, software and/or algo's are deployed it allows the trader to do battle and speed ahead even if it's just for a short while.  Competitive pressures, increasing market volatility, regulatory imperatives, risk mitigation and a host of other challenges are the land mines and roadside bombs on the long and winding road that stall and slow causing re-tooling and re-stocking the ammunition (i.e. algo strategies). There is no time to stop and catch your breath or stand on the roadside.

Sang Lee from the Aite Group reports that High Frequency Trading has had a significant impact on the overall market, providing greater liquidity, tighter spreads and overall improving the quality of the market.  At the macro level these are great advancements and mark a natural evolutionary step due to so many market changes in recent years (i.e. electronic trading venues, adoption of CEP platforms for algo trading, etc.) in Equities and beyond (i.e. FX and Futures & Options).  Down in the trenches, the battles rage on day by day as a multitude of traders and an untold number of algo strategies provide the market liquidity by moving in and out of positions in milliseconds (or even less time). The trading firms engaged in this never ending conflict drive a set of imperatives on software infrastructures for building and deploying algos in the High Frequency battlefield:

Rapid development and customization of algo's

Algo strategies in the High Frequency world have a limited life time. They soon become obsolete (i.e. whatever alpha they took advantage of has disappeared due to the competition, economic changes, or other situations).  To react and respond to this inevitability, having the right sort of tooling to recalibrate strategies is a necessity. This includes graphical modeling tools for Quants to prototype ideas quickly, backtest with historic data, test in a scalable manner to instill confidence prior to production rollout and lastly dynamic parameterization of strategies from graphical dashboards. Not forgetting the code-slinging types, an Integrated Development Environment (IDE) for support of event processing language (EPL) development for more low-level tasks.


Abstracting over increasingly complex strategy logic

Supporting Quants with a rich and robust set of functionality from the basics (connectivity to markets) to the advanced (Linear Algebra, Black Scholes, and other statistical functions).


Support for the 'ilities (availability, security, reliability, ...) to manage the mundane

Deploy with confidence. An important role of software infrastructure is to instill confidence that deployed strategies are always available, securely accessed and run without failure.


Support for scalable performance, providing high throughput and low latency

This is probably the most paramount requirement in the arms race of High Frequency Trading.  The race to the microsecond is pushing both hardware and software vendors alike. Parallelism in CEP engines like Apama's Correlator can leverage multi-core processor architectures like the Intel Nehalem


Along with my colleague Dan Hubscher,  I have recorded a 30 minute webinar that describes how the Apama platform along with the Apama Algorithmic Trading Accelerator meet these imperatives.

The pre-recorded webinar, is available here:  Apama Algorithmic Trading Accelerator, Build Quickly, Run Fast.

Once again thanks for reading (plus watching and listening to the webinar in this case), you can also follow me at twitter, here.
Louie



Tuesday, March 31, 2009

CEP in Transport and Logistics

Posted by David Olson

Now maybe I’m biased since my focus is CEP outside of capital markets but, the announcement of Royal Dirkzwager’s (Koninklijke Dirkzwager  if you’re keeping score in Dutch) use of Apama sets an important tone for CEP. Working with the team that pulled Royal Dirkzwager’s project together, it was easy to see that the business need was there to harness the value of their free-flowing events.

Transport and Logistics (T&L) has a vast fabric of events that range from all the management “systems” to supply chain events and telematics. Without CEP, analysis of those events is either siloed (with spotty real-time abilities) or aggregated a day late and a dollar short. The T&L industry is being squeezed for efficiency and chasing new revenue opportunities is challenging. In the case of Royal Dirkzwager, it didn’t take long for them to survey their domain and recognize that real-time visibility and analysis of events in their fabric could significantly enhance customer satisfaction and the bottom line. Royal Dirkzwager also appreciates that CEP projects aren’t protracted lifetime engagements. They’re looking forward to rapid results.

Royal Dirkzwager’s use of GPS, GIS events as well as information from Automatic Identification Systems (AIS) and Long Range Identification and Tracking (LRIT) systems, when correlated with their other infrastructure events, is where the magic is. Yes, they had all those events before but CEP pulls them together and makes them much more meaningful. They’re also going to leverage their Sonic ESB as a convenient on- and off-ramp for many of their events. And while Royal Dirkzwager’s a maritime logistics provider, it’s not a far stretch to see how other land- or sea-based LSPs can put themselves in a similar position. Whether they own, manage or contribute to the supply chain, the events are there – harnessing them should be an imperative. Royal Dirkzwager’s not our first in this space and it won’t be the last.

I think we’ve done the right thing by having a laser beam focus on capital markets. Truly, a demanding environment when it comes to performance and usability. All that experience gives us the ability to jump into other event-oriented markets with the tools and experience to take on any challenge.

To Royal Dirkzwager, thank you for making my day. Who’s next?

-DO

Monday, March 23, 2009

We're going on Twitter

Posted by Giles Nelson

Louis Lovas and myself, Giles Nelson, have started using Twitter to comment and respond to exciting things happening in the world of CEP (and perhaps beyond occasionally!).

The intent is to complement this blog. We'll be using Twitter to, perhaps, more impulsively report our thinking. We see Twitter as another good way to communicate thoughts and ideas.

We would be delighted if you chose to follow our "twitterings" (to use the lingo), and we'll be happy to follow you too.

Click here to follow Louis and here to follow Giles (you'll need to signup for a Twitter account).