« July 2009 | Main | September 2009 »

August 2009

Friday, August 21, 2009

State of the event processing market, August 2009

Posted by Giles Nelson

The year so far has seen the interest in event processing gathering momentum. It is moving out of being used in niche areas – for electronic trading for example – and being recognised as a general method of building more responsive systems and applications.

There's evidence for this. Two influential analyst firms have published event processing reports this year. IDC in February and most recently Forrester, who published their Wave in July. This means the software market more generally recognises event processing as a distinct discipline. If you follow John Rymer or Mike Gaultieri on Twitter, the two Forrester analysts responsible for the Wave, you'll now see them reporting frequently on customer enquiry calls regarding event processing. Joe McKendrick, influential IT commentator, yesterday described event processing as a bona fide market. In addition, we've just got back the results from a global study in the uptake of real-time technology. We'll be talking more about this soon, but here's an interesting snippet – 86% companies reported that they had critical business events they wanted to monitor in real-time. If that's not evidence that the market for event processing will accelerate in future, I don't know what is. And finally, and perhaps most concretely, the number of vertical industries deploying event processing has increased. In addition to a whole range of use cases in financial services, Progress has now got customers using Apama in telecommunications, transport and logistics, manufacturing, retail and travel.

Why is this happening? Firstly, the benefits of event processing are becoming better understood by a wider group of people, both in the business and technology domains. Secondly, organisations are having to act because of the ever increasing amounts of data being thrown at them. An example of this latter point is Royal Dirkzwager, maritime information supplier, who has taken an event processing approach in part because its data rates are increasing from 10s of events per second to 10s of thousands a second. Finally, organisations generally are having to become more responsive – whether to detect errors or risks with existing business processes or to be able to interact with customers more intelligently. 3 Italia, the mobile telco, provides an example here. The end-of-day report, reporting on what happened rather than what is happening is simply not good enough.

Is it this issue of responsiveness that is really the key driver; the reason why an organisation can commit dollars to a project. Becoming more proactive enables you to spot problems more quickly (and therefore repair issues more cheaply), improve customer service (by, for example, reacting to customer interactions with a Web site or wireless network more quickly) and to be more competitive (for example, by pricing products more dynamically by taking customer propensity and changing market information into account). In every customer I can think of, it is one or more of these issues that has driven them to buy; the understanding of the impact of processing crucial events in the business – of doing business event processing.

The market has yet to become mainstream. It is still one that is about innovation rather than commoditization. It still takes visionaries and light bulb moments for the applications of event processing to be identified. But it is rapidly changing. It won't be too long before the full force of events becomes widely recognised.

Tuesday, August 11, 2009

Flash Trading - To Be or Not to Be

Posted by Chris Martins

The goal of any financial market structure and its supporting rules should be to foster an environment of freedom and transparency where informed market participants can make trading decisions that reflect pursuit of their rational self-interest.  Now truly rational self-interest may be an ideal state that does not exist in the real world as it presumes a trading participant is fully informed and fully rational in their behavior.  Recent economic research (nicely covered in a recent Time Magazine article) suggests entities are rarely fully rational in their actions.  And in financial markets, the participating entities are rarely fully informed because conditions change too quickly and the meaning of the changes is often too opaque to achieve a fully informed state quickly enough.  So participants make educated appraisals with their trade decisions and iteratively appraise and reappraise those decisions as new data is received.

The goal of a regulator should be to try to ensure a reasonably level playing, with emphasis on "reasonable."  It is unrealistic to presume that markets can be perfectly level.  After all, the very term “liquidity” implies movement and fluidity that belies the notion that a perfectly level state can be achieved.  There must be some accommodation in regulatory actions to the natural dynamics of the markets and their participants, much like a sailor must accommodate the movement of the deck in even the calmest sea.  Without such accommodation, we will inevitably stifle the innovation and creativity that are the very foundation of the financial markets.  After all, a ship in dry dock is perfectly stable, but of little use to the sailor.

Regulators should also strive to balance the needs of individual trading entities with that of the trading population as a whole.  After all, one of the foundations of algorithmic trading is the use of technology to minimize order impact.  By mandating, per interpretation of Reg NMS, that orders not avail themselves of a “flash” are we not circumventing the interests of some traders (who wish to disguise their intent), by forcing their trade to be routed to open markets?  They might rationally choose the risk of possible front running vs. the risk of not finding liquidity or the transactions costs associated with that liquidity.  Routing to public markets offers no guarantee of best execution, since the Reg NMS best price standard does not necessarily account for factors like transaction cost or depth of liquidity in these “lit” markets.  Dark pools exist for a legitimate trading purpose.

As regulators look at flash trading and its impact, they should recognize that technology is a tool, neither benevolent nor malevolent.  Those that wield technology in the financial markets will do so in pursuit of their own interests.  That is a good thing because the advantages they seek are often transient and they prompt others to pursue similar advantages with similar vigor.  That vigor is the thriving pulse that has propelled capitalism and made the markets what they are.  The current markets are the better off because of the technology advancements that are currently implemented.  And those advancements were not the result of regulatory demands, but the rational pursuit of self-interest by market participants.   Regulatory restrictions are inherently a blunt tool, best wielded only after careful consideration.

Fair and open markets are a good thing, because they encourage participation in those markets.  Regulators should focus on that goal, because it achievable and because it is an area where the markets (exchanges, trading entities and regulators) all have a common goal.  But expectations must be tempered.  Fairness is not synonymous with perfect equality.  Regulators cannot mandate equal intelligence, equal technology or equal information among all participants.  Some will be better informed, more prescient in their views of the future, or just better equipped. 

As David Byrne might say [or sing], “same as it ever was, same as it ever was!” 

And that is not a bad thing.

Sunday, August 09, 2009

Riding the Crest of the Wave... the Forrester Wave

Posted by Louis Lovas

In just a few short days of its announcement news of the Forrester CEP Wave has spread to all corners of the globe. From trade magazines, online journals and blogs to Facebook and Twitter, the headlines are everywhere. A Google search yields thousands of hits.  "Independent Research names Progress® Apama® as a Standout Leader in CEP ..."

The Forrester team of Mike Gualtieri and John Rymer state 'The Fledgling CEP Platform Market Is Vibrant, Competitive, And Dynamic'. Of course those of us that have been immersed in event processing for the past few years already knew that. It was our job to convince Mike and John. On behalf of Progress Apama and the CEP community, I would like to extend a word of thanks and appreciation to both of them for their efforts, diligence and patience in putting this Wave together. An enormous task given they reviewed 9 CEP products and vendor strategies in depth. Considering this was the first CEP Wave they also had to define an initial blueprint on CEP by which to evaluate vendors, they did a commendable job. Well done. You can get a complimentary copy of the pdf version from us here.  

It was quite a few months ago when I and a few of my esteemed colleagues began the CEP Wave process. In the abstract it was not too much different from responding to the questions in a prospect's RFP/RFI, for which I and my colleagues have much practice. However, a difference that I found unique was the format. A client proposal is generally a Word document where one can provide plenty of written detail, and diagrams to depict product architecture and function. Forrester Waves are MS Excel spreadsheets. Vendor's responses to the Wave's questions are to fit into an Excel cell. Being a long-winded person, it was a challenge to have the necessary succinctness dictated by the confines of a cell.  My colleagues were quite helpful to this end. 

In short order, it became clear as to the benefits of the spreadsheet format. While many documents - proposals, reviews, evaluations or other become static paper the moment they're published that is not the case with Forrester's Wave. There is a clear intent behind Forrester's use of the spreadsheet format; it creates a living/dynamic document for their clients.  Spreadsheet's by their very nature can be interactive. Spreadsheet formula's can accept user input and recalculate. This capability is exactly what Forrester leverages in the CEP Wave.

The Forrester CEP Wave is divided into three categories:
  • Current Offering: A platform feature breakdown, development and deployment tools and performance characteristics.
  • Strategy: The vendors investment for the future.
  • Market Presence: Customer base.
Within each of these categories is an entire litany of subcategories containing features and criteria by which the product and vendor are measured. Each is assigned a weight as deemed appropriate by Forrester in reviewing the CEP industry at large.  Each vendor is then judged by their merits and scored. The most important aspect of this is the weighting. This is the key that gives the Wave that dynamic nature. From a client perspective, the weighting can be adjusted to suit your specific requirements. If for example, your shop is Windows-only you don't need to have a high weight on multi-platform support, you can lower that value. Likewise, if you have strong need for high availability/disaster recovery you can increase that weighting. Making these adjustments will tune the Wave for your specific requirements. You will then see how vendors stack up against each other with your customized weights. By doing so, what you will find is that the Apama platform pops to the top of the list all too often.

Once again thanks for reading, you can follow me at twitter, here.

Tuesday, August 04, 2009

Forrester Wave Cites Apama

Posted by Chris Martins

Forrester Research, a leading independent research firm, has just published its “Wave” on CEP and Progress Apama has been judged to stand out as a leader.  The evaluation process is quite detailed and lengthy.  It addresses:

·        Current product offering: product architecture, features, development environment, administration, interoperability, etc.

·         Vendor strategy: vendor’s product road map and other key strategy elements.

·         Market presence: market presence in terms of customer base, vendor size and presence, etc.

We are very excited to have Apama recognized in this way.  Together with the validation of our 115+ customer implementations (we don’t just score well in evaluations, but also in the real world of customers), this report speaks both to the vibrancy of the market and our leadership position.  But don’t take my word for it.  For a look at the actual report, including the graphical representation and the tabular scoring, you can check it out for yourself here.