Event Processing blog

Sibos 2008 - the event processing angle

I am writing this at the end of the Sibos financial services show in Vienna. Sibos is the biggest global banking event of the calendar with pretty much everyone involved in the core banking area present including commercial banks, central banks, regulators, vendors and consultancies. It couldn’t, of course, have take place at a more interesting time. The extraordinary events we have witnessed this week in financial markets permeated every panel session, presentation and informal discussion held.

Event processing is big in financial services but, so far, it has generally only penetrated the front-office and middle-office risk management functions for use cases related to trading. There are good reasons for this: in general terms the front-office uses technology to directly enable more money to be made. Core banking on the other hand is about doing what banks were set up to do – to take deposits, to give credit and to be an arbiter of financial transactions. The reasons for technology investment are quite different and are driven by increasing operational efficiency, lowering costs and improving customer service. It’s more conservative and as yet event processing has not penetrated this area to any significant extent.

There’s no lack of use cases, I believe. Here are a couple of examples around the processing of payments. Earlier this year the UK launched its Faster Payments Initiative. Finally in the UK (the Netherlands, for example, have had this for 10 years) you can now pay a counterparty who banks with another UK bank in real-time, rather than waiting 3 days for the payment to clear (it’s remarkable it’s taken so long to fix such a, frankly rubbish, state of affairs and indeed it took the regulator itself to force change). As an end-user of this I am delighted with the results. I can now make an electronic payment using Web-based banking and it all happens immediately – the transaction occurs in the way one feels in the modern Internet era that it should. However this does raise a problem: how does a bank do its anti-money laundering checks, its comparison with counterparty blacklists and all the other fraud checks in the 12 seconds it has for the payment to go through? The answer is – currently with enormous difficulties.  Event processing is surely part of the answer.

Here’s another example. Currently the European payments industry is going through a lot of regulatory change to bring about lower prices and more competition for cross-border Euro payments (PSD and SEPA are the relevant acronyms if you’re interested). This will force technology investment because consolidation will mean a smaller number of banks will have to process more payments at lower cost. Furthermore competition will increase and, for example, a business in France will be able to use a bank in Germany to deal with its payments. Now, I reckon that having insight into what is going on with my payment systems, being able to identify processing exceptions, being able to identify when my customer SLAs are being exceeded and so on in real-time will be a crucial part of ensuring a world-class operation. Payment systems will continue to use many different types of technology, from mainframe to modern SOA environments,  so you need something to logically sit above this, extracting relevant real-time information and analysing and correlating it appropriately. There are offerings from conventional BAM vendors that address some of this now but I think they won't be performant or flexible enough to deal with future needs. Some customer engagements support this.

All of this is really about risk management and it seems inevitable that this area is going to be a booming area of investment in the next few years. Knowing what is going on in your business transactions, as they occur, will become more and more important. For example, in electronic trading it is becoming vital to not only regulators and trading venues (such as our customer Turquoise who we announced went live with Apama this week) but also to brokers. They want to know what their customers and they themselves are up to.

I think Richard Oliver, Executive Vice President at the Federal Reserve summed it up well. When asked about the future role of technology in core banking and payment systems he responded that the “immediacy of information is going to be vital” and that it was going to be all about “getting value from the information flows”. I think that provides a pretty good fit for event processing.

Acronym irrelevance

There’s been lots of discussion very recently (and lots of discussion about the discussion) about how CEP is related to other software acronyms and what constitutes a CEP use case or not. See here and here.

This kind of debate depresses me in two different ways. Firstly it displays symptoms of a more general software malaise – the wish to group and pigeon hole things into software classes which then confuse people who are not in the in-crowd. Once a name is agreed upon, let’s say Complex Event Processing, it then gets reduced to an acronym - CEP (excuse the pedantry but this is actually an initialism, not an acronym but that's enough of that). People feel a little sense of achievement. “We’ve made it – we’ve got a TLA!”. Debates then rage about how CEP relates to BRMS, BAM, ESB, BPM, ESP, BEP, EDA and EAI. Dreadful stuff, but, yes I know, we’re all guilty of it at times; it does give others the impression that the IT industry has its head up its own back passage.

The second reason this debate depresses me is that I really don’t understand this constant wish to class things as problems which fit into the "CEP class" or not (just see all the nonsense, albeit amusing, around routers of various types voiced by Messrs Bass and Palmer). Software is ultimately a productivity tool and what end-users really want to know is whether a product will help them achieve something they would be unable to do so by other means. End-users are using and considering using event processing products for a whole variety of purposes – trading, order routing, travel information dissemination, exception management in long-lived IT processes, detection of aberrant conditions in groups of pressure sensors, detection of fraud at retail point-of-sale terminals… the list could go on. People who have a problem to solve might think that event processing technology could help them. It is their responsibility, together with a vendor who may hope to sell something to them, to determine whether a product would help them or not. Were people doing event processing before off-the-shelf products came along? Yes. Do you need a CEP product to do event processing? No. Does everything you do with a CEP product have to involve complex, multiple streams of perhaps temporally related information existing in some distributed computing data cloud? No, of course it bloody doesn’t. I note that Opher Etzion seems similarly turned off by this type of debate.

And before I go I’m quite aware that I’ve used the CEP initialism eight times in this posting. I think I can be excused - this is a Complex Event Processing blog after all.

Real-Time Risk and Surveillance

Last week we heard of Société Générale and its $7B loss. Earlier this week, we at Progress announced that Turquoise, the European Multilateral Trading Facility (MTF), had selected Apama to support its market surveillance and anti-fraud operations (see the previous blog entry for more information on Turquoise). Following our Turquoise announcement therefore, a common question addressed to us has been: "how can technology, and more specifically Apama, be used for risk management and could it have prevented the SocGen loss?"

Specifically on SocGen, considering that the fraud was committed over an extended period of time and by someone well versed in the way that the bank's systems and processes operated, technology by itself certainly couldn't have prevented it. However fraud detection (as our Financial Services Authority and Turquoise customers show) and risk management is a key area where financial organisations have been investing in technology.

Trading and financial markets have accelerated substantially over the last few years. Take the New York Stock Exchange. Between 2001 and 2006 (the last year figures were available), the total number of transactions increased by nearly 400%. Such a change is happening elsewhere in the world. The total number of transactions on the London Stock Exchange for the same period increased by nearly 300%. These increases have been caused both by the total volume of shares traded going up and also by the average size per trade going down. Electronic trading in other products has also increased very substantially - foreign exchange for example.

With markets moving more quickly there is an increased need for organisations to have up-to-date information on their positions and risk exposure. By knowing when limits are breached or when markets have moved unfavourably positions can be adjusted or closed down quickly before things get out of hand. "Up-to-date" often has to mean "real-time" and with the right infrastructure, analysis and reporting technology then real-time really is possible. In particular this is where CEP technology, such as Apama, comes in.

The best known use for Apama in capital markets is algorithmic trading. Algo trading has been a cause, and a response to, the market velocity increases that have occurred. Organisations that have deployed algo trading systems are finding that their use is putting pressure on the middle-office risk function to have a real-time view on trading activity. This has led to the adoption of exactly the same CEP technology to monitor and analyse this trading activity and report in real-time on the associated risk positions. Many of our customers are using Apama for this purpose.

It is this real-time capability of CEP that appeals to both the FSA and Turquoise. As, respectively, a regulator and operator of a market they feel the pressure that comes from the increased velocity of market activity. Conventional market surveillance techniques, where perhaps activity is examined end-of-day or end-of-week are no longer viable. In fact, being able to detect abusive and fraudulent behaviour on a market is of particular importance to ensure the running of a fair market. Often such behaviour takes the form of planned, systematic and repeated interventions which are intended to move the market itself. If you can only find out what occurred days after the event then the markets have already been manipulated and the damage done. By detecting it as early as possible the behaviour can be investigated and stopped.

Can CEP technology used in this way actually create problems? A comment on the FinancialTech Insider blog makes the suggestion that having a real-time view on risk may throw up false positives and thus create more problems. Certainly this is possible, but it should be remembered that technology should always be supportive of a human-centred risk process. Only in the simplest cases will a person not be involved. In the majority of cases further analysis will need to be done and context brought to bear in order to reach a decision about what to do. The value of the technology is to identify issues and potential issues as soon as possible in order for them to be managed in a timely fashion.