Event Processing blog

Sibos 2008 - the event processing angle

I am writing this at the end of the Sibos financial services show in Vienna. Sibos is the biggest global banking event of the calendar with pretty much everyone involved in the core banking area present including commercial banks, central banks, regulators, vendors and consultancies. It couldn’t, of course, have take place at a more interesting time. The extraordinary events we have witnessed this week in financial markets permeated every panel session, presentation and informal discussion held.

Event processing is big in financial services but, so far, it has generally only penetrated the front-office and middle-office risk management functions for use cases related to trading. There are good reasons for this: in general terms the front-office uses technology to directly enable more money to be made. Core banking on the other hand is about doing what banks were set up to do – to take deposits, to give credit and to be an arbiter of financial transactions. The reasons for technology investment are quite different and are driven by increasing operational efficiency, lowering costs and improving customer service. It’s more conservative and as yet event processing has not penetrated this area to any significant extent.

There’s no lack of use cases, I believe. Here are a couple of examples around the processing of payments. Earlier this year the UK launched its Faster Payments Initiative. Finally in the UK (the Netherlands, for example, have had this for 10 years) you can now pay a counterparty who banks with another UK bank in real-time, rather than waiting 3 days for the payment to clear (it’s remarkable it’s taken so long to fix such a, frankly rubbish, state of affairs and indeed it took the regulator itself to force change). As an end-user of this I am delighted with the results. I can now make an electronic payment using Web-based banking and it all happens immediately – the transaction occurs in the way one feels in the modern Internet era that it should. However this does raise a problem: how does a bank do its anti-money laundering checks, its comparison with counterparty blacklists and all the other fraud checks in the 12 seconds it has for the payment to go through? The answer is – currently with enormous difficulties.  Event processing is surely part of the answer.

Here’s another example. Currently the European payments industry is going through a lot of regulatory change to bring about lower prices and more competition for cross-border Euro payments (PSD and SEPA are the relevant acronyms if you’re interested). This will force technology investment because consolidation will mean a smaller number of banks will have to process more payments at lower cost. Furthermore competition will increase and, for example, a business in France will be able to use a bank in Germany to deal with its payments. Now, I reckon that having insight into what is going on with my payment systems, being able to identify processing exceptions, being able to identify when my customer SLAs are being exceeded and so on in real-time will be a crucial part of ensuring a world-class operation. Payment systems will continue to use many different types of technology, from mainframe to modern SOA environments,  so you need something to logically sit above this, extracting relevant real-time information and analysing and correlating it appropriately. There are offerings from conventional BAM vendors that address some of this now but I think they won't be performant or flexible enough to deal with future needs. Some customer engagements support this.

All of this is really about risk management and it seems inevitable that this area is going to be a booming area of investment in the next few years. Knowing what is going on in your business transactions, as they occur, will become more and more important. For example, in electronic trading it is becoming vital to not only regulators and trading venues (such as our customer Turquoise who we announced went live with Apama this week) but also to brokers. They want to know what their customers and they themselves are up to.

I think Richard Oliver, Executive Vice President at the Federal Reserve summed it up well. When asked about the future role of technology in core banking and payment systems he responded that the “immediacy of information is going to be vital” and that it was going to be all about “getting value from the information flows”. I think that provides a pretty good fit for event processing.

CEP - Some Applications within Capital Markets

There is occasionally discussion in the blogosphere around the role of CEP within Capital Markets.  Beyond the online chatter, we also hear via offline discussions that Apama comes up in that context, given our strong (arguably dominant) presence in that vertical.  I'd rather not get embroiled in a debate about what is or is not CEP or what are the historical antecedents of CEP.  Actually I would, but I won't here. Let's just say that there have been suggestions that Apama is really not CEP, because it is an algorithmic trading platform.  Or, on occasion, there is the corollary assertion that algorithmic trading is not CEP and since Apama does algorithmic trading, therefore it is not a CEP product, which is false both in terms of the facts and the logical structure of the argument.  And it goes on - and on.

John Bates recently conducted a series of "audio interviews" that talk about some of the different usages of Apama and CEP within Capital Markets.  They might be illustrative to those who see Apama and CEP solely in terms of algorithmic trading or don't really understand algorithmic trading.  The information is not intended to be deeply technical from either a CEP or Cap Markets perspective, but hopefully provides some introductory context for understanding the real potential for CEP in delivering value within that market - and beyond.

Download Rogue Trading >

http://apama.typepad.com.nyud.net:8090/podcast/1_rogue_trading.mp3

Download Early Adoption of Complex Event Processing >

http://apama.typepad.com.nyud.net:8090/podcast/2_cep_early_adoptions.mp3 

Download Risk Management and Market Surveillance >

http://apama.typepad.com.nyud.net:8090/podcast/3_cep_and_risk_management_V2.mp3
 

SIFMA Retrospective

As a follow-up to my colleague Louis' report on last week's SIFMA show, I thought I'd add some thoughts of my own. My conclusion is that it was the most exciting SIFMA show I have experienced.  While I think attendance was down from previous years, I also think the quality of attendees was up. And for me personally, the excitement of being involved in some industry-moving announcements as well as meeting up with many of my colleagues from capital markets firms, vendors, press and analysts was highly invigorating.

So what were the highlights and take-aways for me?

1. CEP is clearly a theme that is getting a lot of mindshare. So many people said that CEP was a key theme of the show – which is great to hear after many years of working to help define the market. It’s also great to see this add to the momentum so soon after the Event Processing Technical Society was launched. The use cases of CEP are many and varied – and there was a lot of interest and questions around this at SIFMA. We demonstrated on our SIFMA booth 5 different CEP use cases on 5 different pods - algorithmic trading, smart order routing, managing FX market fragmentation, market surveillance and real-time bond pricing. Also the demands of CEP applications continue to make demands on the technology, and we were thrilled to demonstrate Apama 4.0 – which extends performance and user experience of CEP to new levels. Another supporting factor in the maturing of CEP is that there are starting to be very senior people in Capital Markets firms focusing on CEP as an enabling technology. Marc Adler from Citigroup   is a key example. He’s active in the community and on the STAC CEP committee, helping to define benchmarks. It was great to meet Marc at SIFMA and also to catch up with many  other esteemed colleagues from the CEP space.

2. The liquidity wars are hotting up. It was our pleasure to be involved in a press release with NYSE-Euronext which was certainly one of the big releases of the conference. Progress Apama will be hosted in the NYSE Euronext as part of the exchange's Advanced Trading Solutions offering. Traders will be able to download custom logic for algorithmic trading, risk management and smart order routing into the NYSE itself - with low latency connectivity to other trading venues via Wombat and Transact Tools. This arrangement turns NYSE into a technology provider as well as a one-stop-shop liquidity provider. This announcement was picked up by major press, including the Financial Times - in Europe, America and Asia -- see the article here.

3. Hardware is important – and so is “green”. The increase of capital markets data volumes require completely new software architectures – like CEP. But software is not always enough to support the low latency transport , processing and storage requirements. Many firms are turning to specialized hardware, combined with software – to create high performance solutions. Vhayu, for example, launched Squeezer – which combines hardware and software to supercharge their tick data offering. Also, Progress Apama were pleased to put out a joint announcement with Sun on a collaboration for end-to-end CEP solutions – combining Sun hardware and operating systems with Apama’s CEP platform and solutions. We demonstrated an end-to-end bond pricing application using the whole stack. Sun was one of the vendors who have a “green” aspect to their hardware – for example on a major CEP deployment, the hardware can be scoped for peak throughput – but can selectively shut down capacity to save power when event throughput is reduced. In this era of high energy costs and global warming there seems to be a lot of interest in this approach.

4. I love partying on the trading floor. Progress Apama were honored to be invited to a party at the NYSE to celebrate the latest developments at NYSE-Euronext (see picture at the top). It was a great pleasure to speak with our friends at NYSE-Euronext and to meet many of our old friends from the capital markets industry there – while sipping some delicious wine in that amazing place. In a way it is a shame that electronic trading is making the traditional trading floor a thing of the past – but there is something amazing about that place and I hope it stays just the way it is – even if it becomes a cool venue for other purposes. Thanks to NYSE-Euronext for inviting us – we had a great time.

I’m sure I’ve neglected a load of other trends and themes – but there’s my brain dump for the day. I’m interested to hear if you all agree.

John

Asia Report: Fighting White Collar Crime

Hello from Hong Kong. As always it is fascinating to see how CEP is evolving in Asia. One trend I am observing is the huge interest in Hong Kong in rogue traders and white collar crime – and how CEP can be used to detect and prevent this – before it moves the market. Obviously the original rogue trader, Nick Leeson, is well known here. But there has been a great deal of interest in more recent goings-on, at firms such as SocGen. Amazingly, until a couple of years ago, insider trading was not illegal in Hong Kong! Now we have a highly volatile market, with a lot of uncertainty, huge event volumes and a real problem of seeking out and preventing rogue trading activities, as well as managing risk exposure proactively.

Of course CEP provides a compelling approach. In market surveillance - the ability to monitor-analyze and act on complex patterns that indicate potential market abuse or potential dangerous risk exposure can allow a regulator, trading venue or bank to act instantly. Banks want the reassurance that they are policing their own systems. Regulators need to protect the public. The media and public here find this fascinating.

On the topic of a different kind of white collar crime – consider using CEP to detect abuse in the gaming industry. The gambling phenomenon that has propelled Macau to overtake Las Vegas as the world’s biggest gambling hub is also an exciting opportunity for CEP. We have customers using CEP to monitor and detect various forms of potential abuse in casinos. Events that are analyzed to find these patterns include gamblers and dealers signing on at tables, wins and losses, cards being dealt etc. It is possible to detect a range of potential illegal activities, ranging from dealer-gambler collusion to card counting.

As a final thought - having met with some of our customers that operate both in Hong Kong and mainland China, it is clear that China is a massive market opportunity for CEP. Exciting times ahead for CEP in Asia.

Real-Time Risk and Surveillance

Last week we heard of Société Générale and its $7B loss. Earlier this week, we at Progress announced that Turquoise, the European Multilateral Trading Facility (MTF), had selected Apama to support its market surveillance and anti-fraud operations (see the previous blog entry for more information on Turquoise). Following our Turquoise announcement therefore, a common question addressed to us has been: "how can technology, and more specifically Apama, be used for risk management and could it have prevented the SocGen loss?"

Specifically on SocGen, considering that the fraud was committed over an extended period of time and by someone well versed in the way that the bank's systems and processes operated, technology by itself certainly couldn't have prevented it. However fraud detection (as our Financial Services Authority and Turquoise customers show) and risk management is a key area where financial organisations have been investing in technology.

Trading and financial markets have accelerated substantially over the last few years. Take the New York Stock Exchange. Between 2001 and 2006 (the last year figures were available), the total number of transactions increased by nearly 400%. Such a change is happening elsewhere in the world. The total number of transactions on the London Stock Exchange for the same period increased by nearly 300%. These increases have been caused both by the total volume of shares traded going up and also by the average size per trade going down. Electronic trading in other products has also increased very substantially - foreign exchange for example.

With markets moving more quickly there is an increased need for organisations to have up-to-date information on their positions and risk exposure. By knowing when limits are breached or when markets have moved unfavourably positions can be adjusted or closed down quickly before things get out of hand. "Up-to-date" often has to mean "real-time" and with the right infrastructure, analysis and reporting technology then real-time really is possible. In particular this is where CEP technology, such as Apama, comes in.

The best known use for Apama in capital markets is algorithmic trading. Algo trading has been a cause, and a response to, the market velocity increases that have occurred. Organisations that have deployed algo trading systems are finding that their use is putting pressure on the middle-office risk function to have a real-time view on trading activity. This has led to the adoption of exactly the same CEP technology to monitor and analyse this trading activity and report in real-time on the associated risk positions. Many of our customers are using Apama for this purpose.

It is this real-time capability of CEP that appeals to both the FSA and Turquoise. As, respectively, a regulator and operator of a market they feel the pressure that comes from the increased velocity of market activity. Conventional market surveillance techniques, where perhaps activity is examined end-of-day or end-of-week are no longer viable. In fact, being able to detect abusive and fraudulent behaviour on a market is of particular importance to ensure the running of a fair market. Often such behaviour takes the form of planned, systematic and repeated interventions which are intended to move the market itself. If you can only find out what occurred days after the event then the markets have already been manipulated and the damage done. By detecting it as early as possible the behaviour can be investigated and stopped.

Can CEP technology used in this way actually create problems? A comment on the FinancialTech Insider blog makes the suggestion that having a real-time view on risk may throw up false positives and thus create more problems. Certainly this is possible, but it should be remembered that technology should always be supportive of a human-centred risk process. Only in the simplest cases will a person not be involved. In the majority of cases further analysis will need to be done and context brought to bear in order to reach a decision about what to do. The value of the technology is to identify issues and potential issues as soon as possible in order for them to be managed in a timely fashion.

Apama Wins Market Surveillance Deal at Turquoise

Progress Apama, along with our system integration partner, Detica, has been chosen by Turquoise, the European multi-lateral trading facility, to deliver a real-time market surveillance system. Turquoise is funded by nine of Europe's largest banks and is essentially a new electronic stock exchange that, when it goes live this year, will compete with traditional equities venues like the London Stock Exchange.

Explicit in the announcement is how important market surveillance can be in helping to ensure an orderly market that traders can trust to be fair to all participants. Such confidence helps promote liquidity, which is the lifeblood of any trading venue. Implicit in the announcement is the value of CEP-powered, real-time market surveillance in driving the rapid detection of potential patterns of abuse or other questionable behavior. As Apama observers will note, this is the second selection of Apama and Detica for a real-time market surveillance system.  Last year, the same vendor team was chosen by the British regulator, the FSA.

There has been much press in the last week regarding the problems that can occur when proper procedures – and technology – are not in place to monitor trading behavior.  The situations are different - the problems at the French bank Société Genéralé appear to involve the Futures trades of a single trader, while Turquoise will be an equities exchange. But billions in losses certainly shine a bright light on the value of being able to monitor activities and respond quickly, regardless of the financial instruments involved. And though the motivations might differ, the value is there whether the monitoring is done by individual firms, by the trading exchanges, or by regulatory authorities.

Dr. John Bates on Fox Business News

As readers of this blog know, Apama was chosen by the UK regulatory agency, the Financial Services Authority, to provide the CEP technology that delivers real-time market surveillance as part of the the FSA's SABRE II project (FSA/Apama Announcement). In a recent interview by the Fox Business Network, John Bates discussed how Apama can be used to detect patterns of suspicious trading behavior - much like Apama is so often used to identify and respond to market patterns in support of algorithmic trading applications.  The link below will take you to the Fox URL that plays the video.

Fox Business Interview with Dr. John Bates