« November 2008 | Main | February 2009 »

January 2009

Tuesday, January 27, 2009

Calling all VARs

Posted by David Olson

Greetings from Raleigh on January 26, 2009

I suppose an introduction is in order. My name is David Olson. Long time Progress person and I tend to float between technical and business roles in the company. Recently crossed over into the Apama world to focus on opportunities other than Capital Markets, which brings me to an interesting observation…

I've spent a lot of time working with our Application Partners (formerly known as VARs) and over the past few years I've made an effort to push CEP concepts to key partners. Many are in industries that have a few event streams that are ripe for the picking. Manufacturing, logistics, telco, and retail certainly rise to the top of the list. All tend to agree that there are events streaming through their customer's infrastructure and there may be a potential use case there but, few know what to do about it - and I suppose that's understandable.

For 50 years or so, we've concentrated our business application development on transforming traditional business processes into software. MRP/ERP has certainly led the call – but for the most part, ERP is done. And like the Holy Grail in the Monty Python movie it's getting harder to sell ERP because, you see, we’ve already got one.

Operational Effectiveness through CEP is the next frontier for business application software and I believe that it will shake up business processing the same way ERP did in the past. This isn't to say that ERP is not necessary, it's just that underlying platforms are not conducive for the "in the moment" nature of CEP solutions. In fact, the two should live comfortably side-by-side.

So VARs, this is your call. The infrastructure is mature, the technology is available, and your customers are ready. There's a new seed you can plant that can bring additional opportunity to your existing solutions. Take a look at what Manuvis has done with their FactoryMRI solution set. Real-time dashboards - right on their home page. CEP's a big part of what they do and they're making waves.

You should be next.


Friday, January 23, 2009

The World According to a 4-Year-Old, and the Real Business of Foreign Exchange

Posted by Dan Hubscher

The following conversation with my 4-year-old daughter happened just a few days ago:

Her:    I really like China, it sounds like a great place.

Me:     Why?

Her:    They have the best toys in the world there.

Me:     How do you know?

Her:    All my favorite toys are "Made in China."

I stopped asking her questions.  But my mind quickly went from wondering about the origins of toys to wondering about the origins of the capital markets business applications I encounter these days.  These applications rely heavily on functions like alpha-seeking and execution algorithms, and foreign exchange aggregation, to name a few.  I’ll elaborate on others in future posts, but the point is that this is a high risk environment, not child’s play.  So where do the ideas for those functions come from? 

A few Apama customers I have met lately note that the markets seem to be acting like a bunch of 4-year-olds having temper tantrums.  They noted wild swings in the FX marketplace, for instance, since the credit crunch became a crisis a few months ago.  FX trading will continue to be frantic this year. But necessity is indeed the mother of invention, which is a good thing with the markets the way they are.

This week, we announced that Standard Chartered Bank has gone live with the Progress Apama FX Market Aggregation Accelerator.  We’ve talked about FX aggregation before, and have done a number of similar Tier 1 deployments.  As you would expect, some great ideas in these projects have been customer-driven.  And when I look behind the scenes at Apama, I see how these projects provide a springboard for people to think of new things and try them out.  

On my very first day at Apama, a new colleague walked up to me and said, “I solved a problem that was bugging me.  Do you think customers might like this too?”  These ideas tend to grow into a demo, which may become the basis for a collaborative deployment, and eventually a customer has built on the idea and taken it in a new direction – a similar Apama story can be found here

Yesterday someone asked me if I think that Capital Markets firms are using technology to save cost or to grow.  Cost savings?  Yes, at least, and for some that may be the end of the story and the mind is closed.  But a former manager of mine, now a colleague and adviser, taught me to listen to the markets by telling me, “You have 2 ears and 1 mouth, use them in that proportion.”   A similar philosophy serves software firms, as well.

Funny that this week, one of my Wall St. friends at a very alive firm wrote to tell me his opinions about the state of software evolution, and the importance of innovation.  I think the crisis will force patches of growth and innovation.  Do you?  Comments welcome.



Monday, January 19, 2009

A Transparent New Year

Posted by Louis Lovas

It's been a while since I've put a few thoughts in print on these pages on event processing, but that's not to say I've not been busy. We've wrapped up quite a successful year in the Apama division within Progress software. For this year, 2009 we've got quite a number of new features and capabilities in store for the Apama CEP platform. We will be extending the breadth and depth our event processing language (EPL) to meet the challenges we've seen in the market and a vision we have for the future. Of course you will see announcements in our standard marketing press releases but I (and others) will explore the technical merits of those new features in these blog pages in much more detail. Something we've not done that much of in the past. There are many historical reasons for that not really worth explaining. Suffice to say our intention is to be more transparent in the coming year.

To kick that off, it's worth starting a bit of a tutorial on the Apama EPL, a language we call MonitorScript.  I'll begin with the basics here and in subsequent blogs build upon these main concepts, providing insight into the power and flexibility of our EPL. And as we release new extensions and capabilities it will be easier to explain the benefits of those new features. So without further ado, here is the first installment.

First a few basic concepts...

  • Apama MonitorScript is an imperative programming language with a handful of declarative statements. This is an important consideration and one we highlight as a distinction in our platform against competitive platforms that are based on a declarative programming model. The imperative model provides a more natural style of development similar to traditional languages like java and C++.
  • The language is executed at runtime by our CEP engine called the Correlator.  

Second a few basic terms...

  • A monitor defines the outer most block scope, similar to a class in java or C++. It is the basic building block of Apama applications. A typical application is made up of many monitors. As you might imagine monitors need to interact with one another, I'll explore that capability in a later blog.
  • A event defines a well ordered structure of  data. The EPTS Glossary definition has a handful of great examples of Events
  • A listener, defined by the on statement, declares or registers interest in an event or event pattern. In a large application it's the Correlator runtime engine that will typically process 1,000's or even 10,000's of registered event patterns. In our example below we just have one.
  • An action defines a method or function similar to java or C++. 
  • The onload() action is a reserved name (along with with a few others) that is analogous to main() in a java program. 
To put it all together... "A monitor typically listens for events and takes one or more actions when an event pattern is triggered."

The language sports a number of capabilities that will be familiar to anyone schooled in java, C++ or any traditional imperative programming language. I won't bore with all the nuances of data types and such obvious basics, those are well articulated in our product documentation and customer training courses. I will however, focus on the unique paradigm of event processing.

Apama MonitorScript, a basic monitor.

event StockTrade {
  string symbol;
  float price;
  integer quantity;

monitor StockTradeWatch {

  StockTrade Trade;

    action onload {
      on all StockTrade():Trade processTick;

    action processTick {
      log "StockTrade event received" +
      " Symbol = " + Trade.symbol +
      " Price = "  + Trade.price.toString() +
      " Quantity = " + Trade.quantity.toString() at INFO;

This monitor called StockTradeWatch defines an event of type StockTrade. Events can originate from many sources in the Apama platform, the most obvious would be an adapter connected to a source of streaming data (i.e. stock trades as example shows), but they can come from files, databases, other monitors, even monitors in other Correlators.

The onload action declares a listener for events of type StockTrade (i.e. on all StockTrade). When the monitor StockTradeWatch receives StockTrade events, the action processTick is invoked. As you can see in this example we simply log a message to indicate that it occurred. The obvious intent is that this monitor will receive a continuous stream of StockTrade events, each one will be processed in-turn by the processTick action.  One can be more selective in the event pattern with a listener, such as on all StockTrade(symbol="IBM"). I will explore the details of event patterns and complex listeners later on.

As I mentioned, I've started with a simple example that shows the basics of the Apama EPL, MonitorScript. It demonstrates the simplicity by which one can declare interest in a particular event pattern, receive matching events and act on them in your application (i.e. action).

 In subsequent installments I will demonstrate more complex features highlighting the power of the language.  That's all for now.

You can find the second installment here.

Wednesday, January 14, 2009

And the winner for the best CEP product goes to....

Posted by Giles Nelson

IDC, one of the "big three" software analyst firms has just published a report on CEP - Complex Event Processing Opportunity Analysis and Assessment of Key Products (more information about it is available here).

The report has a few major themes. Firstly, IDC look at the market for CEP applications. They state that trading applications in capital markets makes up the biggest single CEP application segment but go on to note that it represents only a minority of CEP applications. Other areas mentioned where CEP is in production are risk analysis, fraud identification and general transaction & business process monitoring in a range of different industries. Transportation and logistics is cited as one particular industry where CEP is gaining significant traction. IDC also mention use cases in a whole range of different verticals where deployments of CEP are live: healthcare, government, leisure, banking, media and manufacturing; this breadth of application areas is excellent validation that CEP is not a technology only useful within the capital markets trading arena.

In terms of market growth IDC believe CEP will continue to grow strongly, despite the general market downturn. In their view, one which I share, the increased focus on risk management and on running existing operations more effectively and efficiently will actually further stimulate the CEP market. IDC estimate that CEP is the biggest growing segment of, what in their categorisation is, the middleware market.

Thirdly, from a parochial Progress point of view, here's the best bit: as well as looking at the CEP market and where CEP is being used, IDC also did a vendor evaluation using various criteria covering correlation sophistication, integration with event sources, the presentation of results and control of event processing scenarios by business users etc. All the usual suspect vendors participated. I am very pleased to say that Apama came top which, of course, is very gratifying.

My final point is that what's most important here is not the views of any one analyst firm, but that the publishing of such a report validates that CEP is becoming more important to end-users. Analyst firms are driven by end-users and thus the fact that an increasing number of analyst firms are investing in conducting CEP research demonstrates that they believe there's an end-user market for it. And end-users are the most important people around.