Event Processing blog

Bringing Order to Machine-led Chaos

Editor’s Note: the following post is written by Theo Hildyard, Solutions Architect, Market Surveillance at Progress Software was origianally published on TABB Forum.

Since the May 6, 2010 flash crash the issue of out-of-control machines trading on global stock markets has made headlines over and over. Just last week a US market maker, Knight Capital, nearly blew itself up with a rogue algorithm. The calls for regulation are growing louder and regulators globally are struggling to bring order to an automated marketplace that is increasingly chaotic.

In India, the Securities and Exchange Board is considering imposing speed limits on high-frequency trading. The Hong Kong Securities and Futures Commission's CEO is very keen to regulate HFT and proposes that algorithms are tested annually. Australia's Securities and Investments Commission (ASIC) wants automated trading systems tested. In Europe the Securities and Markets Authority (ESMA) is preparing to crack down on every aspect of automated trading from algorithms to CDS to short selling. And in the US the Securities and Exchange Commission is tightening rules on automated trading systems and programs, with Knight Capital having added to the urgency.

Machines trade anywhere from 25% (Australia) to 70% (US) of the volume on stock exchanges. The opportunity to make money depends upon the speed of your trading systems along with the intelligence of your algos. Algorithmic innovation is critical in order for high frequency trading firms to find an edge. Research done by the AITE Group suggests that the average lifespan of a trading algorithm can be as short as three months. With such a small window of opportunity trading firms must design, test and deploy new algos on an almost continual basis. No wonder there are problems.

When we allow machines to make the decisions for us, it is imperative that we design them to be fail-proof. Testing in a non-production environment should be mandatory, and back-testing should be exhaustive. Poor quality due diligence and quality assurance is producing catastrophic consequences. It is our responsibility to ensure that our machines, or robots if you will, do no harm.

I am reminded of author Isaac Asimov's first law of robotics: "A robot may not injure a human being or, through inaction, allow a human being to come to harm." The 'harm' in Rule #1 is happening to the marketplace. The flash crash wiped a trillion dollars off of US-listed firms' market capitalization. Knight Capital's rogue algo wiped $440m off its balance sheet and forced it to look for backers in order to survive.

If algorithms and trading systems were programmed with Asimov-style parameters, there would be far fewer glitches. But even if you are the most conscientious firm out there, you cannot ensure that your counterparties have also programmed and tested their systems and algos thoroughly. And it remains your responsibility, to your customers, staff and shareholders, to ensure that those counterparties do not do any harm to your bottom line or reputation.

Catastrophe can only be avoided by adding an extra layer of control in the trading process; a layer which monitors counterparties for rogue algos or fat fingered trades. That way you have both belts and braces - control over internal trading systems and awareness of external ones. Yes, there will be a tiny bit more latency. But isn't a small latency hop better than bankruptcy? 

 

Scooping FX Bubbles Out of a Boiling Pot

Foreign exchange trading appears to be moving from a beneath-the-radar, bank-dominated activity into the international trading limelight. There has been an explosion of new trading platforms and a wave of newer participants lately, partly thanks to new transparency afforded by Dodd-Frank and partly because of the relentless hunt for alpha. Increasingly automated, FX is also becoming the next go-to asset class for high frequency and algorithmic trading.

It wasn't always that way. As TABB Group's Larry Tabb said in an article in Wall Street & Technology: "FX has always been different. Be it that currency is a bank’s core product, be it that banks control the payments infrastructure, or be it that banks are critical in implementing Central Banks’ monetary policy, the banks have historically dominated FX."

Because FX is mainly traded via single dealer platforms, multi-dealer platforms such as FXall, and interdealer marketplaces, it is fragmented in a different way from equities.  Traditional trading platforms along with a couple of the sturdier newcomers like multi-dealer platforms FXall (which Thomson Reuters is buying) andCurrenex have been the dominant destinations for electronic trading of FX.

But now that the SEC and CFTC have clarified that forex contracts will be determined to be swaps, they will become part of the centrally cleared instrument pool. This means a whole new layer of banks, brokers, and venues are already popping their heads up. FX will soon emulate the expansion, consolidation, and then contraction of destinations experienced by the equities markets. 

There will be more opportunities for market participants to trade, hedge, arbitrage and manage risk. Algorithmic strategies will dominate, attracting more and more destination venues - and then fragmentation will be the mantra. So how are traders going to position themselves to scoop the profitable FX bubbles out? It is not as easy as you would think. In many cases, what appears to be an increase in liquidity is actually an increase in “phantom” orders, as institutions advertise the same underlying liquidity across an increasing number of locations. Trading algorithms will need to be smarter, and tuned over time to counteract this as the landscape changes.

Bank traders are looking for ways to handle the new world order of FX. Because their clients want to be able to trade forwards, swaps, spot and even options on the same system, banks are having to do the once-unthinkable: merge their forwards desks with their spot desks.

In the old days of voice trading, forwards and spot traders ran completely separate books and dealt with (mostly) different customers. Today clients are asking to hedge forwards and spot on the same system at the same time. Some want to trade using forward-to-spot conversions against aggregated spot prices from several platforms and some want to use aggregated forward rates directly. Some want a blending of both. The opportunities for banks are plentiful, if they can harmonize FX products, trading and hedging across trading systems successfully.

Many bank clients have seen what has happened in the equities markets; with high frequency trading and algorithmic strategies becoming problematic and largely vilified. When the world’s largest interdealer brokersaid recently that it would tackle “disruptive” practices by high-frequency traders on its foreign exchange platform it became clear that some of the lesser-loved equities issues were already creeping into FX markets.

The Wall Street Journal says that there are already fears of an FX "boom" reminiscent of the equities venue explosion in 2001. "Some market insiders fear the trend for highly specialized new systems aimed at separate pockets of clients could end up splitting the liquidity that underpins this $4 trillion-a-day market, making it harder to trade," said the paper. Pigeon-holing traders, whether it be by class of trader, asset class or by delivery date, only creates more fragmentation. This could equate to lower volumes (like equities), more volatility (like equities) and an increase in manipulative practices (like equities). Regulators will no doubt be watching, and new rules will be implemented even faster than has happened in equities.

In a discussion at the FX Week USA event in NYC recently one FX trading platform provider said that you need a full market ecology to provide proper efficiency. This means having all market participants operate in the same liquidity pool.  In the end the unique self-regulating properties of the FX markets mean that the market will shift towards what is best for the market - because it can. Preparation for this inevitability will determine who survives. 

Therapy for Toxic FX Order Flow

As high frequency and algorithmic trading infiltrate foreign exchange markets some of the problems that dog equities, such as high order cancellations, are arising.

Equities markets, which have seen HFT and algo trading go through the roof, have recently started clamping down on excessive and cancelled orders. As my colleague recently explored, Deutsche Börse, Borsa Italiana, NASDAQ and Direct Edge have all announced intentions to discourage the number of cancelled orders they receive. They will encourage the "good" liquidity, those players with high fill ratios, and punish the "bad".

The IntercontinentalExchange has already seen good results from a policy it implemented last year aiming to discourage "inefficient and excessive messaging without compromising market liquidity." Regulators, too, are taking note; the SEC is considering charging HFT firms for cancelled trades.

A combination of economic incentives and controls makes it happen.  In addition to adjustments to their rebate schemes, exchanges must monitor their market makers in real-time to make sure that they are living up to their quoting obligations. This monitoring can also include spotting the "Stupid Algos" blamed for generating a burden the exchanges cannot bear. 

It was only a matter of time before other asset classes started to see similar problems with excessive orders, and a similar response via a new generation of intelligent “sensing” algos – but with a twist.

FX is increasingly traded by computers.  Consultancy Aite Group said in a report last year that FX algorithms will account for more than 25% of FX trade volume by the end of 2014. And as algorithms take control, the opportunity for a flood of quotes and cancellations increases. Order-to-trade ratios, the number of orders that come in compared with the number filled,  create a load on exchanges and electronic markets and they can provide a smokescreen to hide potentially abusive behavior (so-called “quote stuffing”).

We see innovative FX brokers taking measures to rein in unproductive order flow.  Similar to equities marketplaces, FX dealers and brokers are increasingly utilizing tactics that discourage excessive orders, but in a very different way. Because FX is mainly traded via single dealer platforms, multi-dealer platforms such as FXall, and interdealer marketplaces, it is fragmented in a different way from equities. 

So it is the FX brokers that are acting like exchanges and taking the initiative to control toxic order flow with their pricing strategies. Brokers need to see every opportunity and threat hidden in their customers’ flow patterns, and automate their own real-time responses, to stay profitable as markets change. Brokers servicing HFT clients react to predatory algorithms and fluctuating fill ratios by manipulating the spreads they offer.  Traditional customer profiling based on purely historical data is good for strategic decision-making.  But for more tactical decisions with immediate impact,real-time analysis is additionally required. 

A responsive broker can, for example:

• Mitigate "toxic flow" by detecting predatory patterns in real-time, and automatically widening spreads to those clients
• Increase business by detecting reduction in flow from “good” clients, and automatically reducing spreads to those clients
• Preserve the relationship by detecting pending credit breaches, and immediately calling the client

Our customers use the Apama platform to perform their own customer flow analysis.  Both global and regional FX brokers now optimize how they serve their customers based on detailed real-time diagnosis of their flow. Key parameters include P&L on individual trades, an aggregated view of individual trades over time, and the performance of tiered client groups.  Using real-time customer flow analysis brokers (and banks and trading platforms) can figure out which customers are providing the types of order flow that they need. 

Customer flow sits alongside other real-time market trend analytics such as volatility, average daily volume, and depth of book.  For example, flow from a specific customer is high but liquidity is thin - then time of day impacts spreads in addition to customer behavior.  Our customers have also been generating pricing dynamically – adjusting spreads and skews - based on market conditions and customer trading patterns – including HFT patterns.

Dynamic pricing builds on an aggregated order book as source pricing.  A basic pricing service dynamically applies a set spread to the base price generated from the aggregated book.  A more advanced service changes the spread based on any data or rule, for example:

• Current volatility
• Depth of book (volume on bid/ask side)
• Real-time risk parameters such as profit/loss levels
• News
• Current vs. target position (changes the spread or skew automatically, and updates the auto-hedger service)
• Customer tier
• Historical & real-time customer trading behaviour

Brokers can take input including aggregated FX prices, customer trading patterns, market volatility and hedging activity - all in real time - into the platform. The analysis generates dynamic pricing (spreads/skews) and it can work to incentivize market participants to provide quality - not quantity - orders. 

Toxic order flow, like excessive orders-to-trade, can tax trading systems and create an environment where fraud and market abuse can flourish. Using real-time customer flow analysis to get a handle on your customers' order flow will help to prevent this. Customer flow analysis can be used not only for dynamic pricing, but also for customizing product offerings and enabling banks and brokers to create execution algorithms for their clients to use. By being proactive, FX brokers and banks can avoid the issues that plague equities. And make money along the way.

 

Déjàvu all over again?

It is fair to say that High Frequency Trading (HFT) is a divisive subject at the best of times; for every expert claiming that it benefits markets in the form of liquidity provision, tighter spreads etc, you can always find another who claims that it poses significant dangers and creates a 2-tier market. Whatever the truth, it appears that there are an increasing number who subscribe to the latter point of view, with the aim of excluding HFT from the market altogether.

I wrote previously about recent declines in trading volumes as an indicator that the HFT “backlash” is having effect. HFT is the unpopular kid in the class no-one wants to sit next to. Witness the recent spate of announcements of new venues that explicitly exclude or penalize HFT and its practitioners. In the FX space we’ve heard about

Mako FX’s plan to build what it calls the fastest trading platform in the wholesale FX market, and more recently the launch of a new venue called FXSpotStream backed by 6 major FX banks who will also be the primary liquidity providers. 

This all gives a real sense of déjà vu, bearing in mind that the EBS FX market was started by a bunch of banks to provide a private inter-bank market, before they let the sharks in and ruined the party. It seems that EBS themselves are now having second thoughts. This highlights something I’ve been saying for some time with regard to HFT; namely, that the market is well equipped to take corrective action if participants care enough, without knee jerk recourse to poorly thought-through regulation. Commercial imperatives will force balancing actions once the pendulum swings too far. This trend is not confined to the FX markets – see CA Chevreux’s launch early this year of Blink, a Dark Pool for European Equities that excludes HFT.

Besides excluding or penalising the HFTs, another "balancing action" I'm seeing is the rapid rise of smart FX execution algos. Our customers have been using traditional VWAP and Percent of Volume style Algos with our Progress Apama FX eCommerce solution for some time, but more recently customers have been telling me how they've had to adapt these algos and build more sophisticated variants, to avoid signalling risk and defeat the HFTs.

This trend to smart algos follows closely what we've seen in the equity and exchange-traded futures markets previously. If it seems like we're re-treading old ground here than that's hardly a surprise - fashions come and go.

But right now it certainly seems that HFT is rapidly running out of friends to play with.

Tune in to our P&L Webinar  “FX Aggregation without the Aggravation” tomorrow to hear more. If you're attending P&L's 2012 Readers’ Choice Digital Markets Awards and Hall of Fame dinner in NYC on Thursday May 31, stop by the Progress Software table to learn more about the Apama FX eCommerce solution. 

The Rat Race to Regulate High Frequency Trading

 

The following is an excerpt from Dr. John Bates’ recent commentary on Huffington Post, whch discusses the current state of high frequency trading regulation.

As Aerosmith famously sang: "Rats in the cellar... losin' money, getting no affection." Lately, HFTs have been compared to everything from rats in a granary to highway robbers intent on stealing Granny's pension. Bashing high frequency trading firms has become the latest sport in the financial services industry. So much so that the Futures Industry Association has publicly taken exception to the "emotive language" being assigned to HFTs.

"For example, many people don't realize that market abuse -- as well as being morally reprehensible -- comes at a hefty price for the market. So principal trading firms such as our members have a very real economic incentive to fight market abuse and back regulatory reform," said FIA European Principal Traders Association chairman Remco Lenterman. He noted that the industry's critics chose to overlook the value that principal trading firms add to the real economy in terms of lower transaction costs and greater liquidity, according to Finextra.

Read the full post from Dr. Bates here. 

 

Automated trading restrictions: are they a presumption of guilt?

Anyone who’s seen the news in recent months will know that High Frequency Trading is facing a sharp increase in the number of regulatory challenges, with some tough measures suggesting that it has been presumed guilty until proven innocent by many. ESMA, implemented in Europe in May 2012, is the latest set of regulatory guidelines around the systems and controls required in an automated trading environment. But are these regulations fair?

It seems clear that, with the ever-increasing volumes of data that firms need to manage and monitor in order to catch abuse, Europe has decided to take a firm stance on automated trading. But is all this a case of, as my colleague Richard Bentley suggests, using a sledgehammer to crack the nut?

Clearly, increasing red tape will place a significant burden on firms and may, if we’re not careful, lead to a situation of regulatory arbitrage, or lock those without deep pockets out of the market. Perhaps a better answer is to adopt a three-layered approach to surveillance where the brokers, trading venue and regulators all have a different role to play will help stamp out abuse without necessarily stubbing innovation?

On a recent visit to London, I met with Phillip Stafford at the Financial Times Trading Room to discuss EU market abuse regulations as can be seen in the video here.

 

Today in Event Processing

In “Cracking the High Frequency Trading Nut”, Richard Bentley discusses the effectiveness of the new guidelines from the European Securities and Markets Authority.  He compares the guidelines to a sledgehammer, questioning if such extreme measures are necessary to regulate HFT.  Would a more precise approach, which targets specific issues with HFT and offers real-time surveillance, better regulate automated trading? Find out in yesterday’s post. 

Today in Event Processing

In his blog post “BRICS Win by Coming in Second”, Richard Bentley explains how emerging markets benefit by coming in second when it comes to high frequency trading. By looking at the first-comers, namely the U.S. and Europe, Bentley highlights how regulators were not prepared, resulting in unsafe practices. 

 

Today in Event Processing

“Solving the Cross-Market Surveillance Conundrum”, from Theo Hildyard, offers his thoughts and insights on the World Exchange Congress, where he discussed the MiFID directive. Although MiFID was created to protect customers in investment services, the market has been manipulated and abused since the directive was first announced in 2007. Based on his conversations with other attendees at the April event, Theo gives his thoughts on how to better regulate and protect the market. 

Today in Event Processing

In his most recent post, Dr. John Bates discusses the technology glitch that caused BATS share price to plummet within minutes of its first IPO. Comparable to an “own goal” blunder, Dr. Bates refers to the malfunction as a wakeup call, urging exchanges and trading destinations to perform extensive back-testing and market simulation to prevent such a problem from occurring in the future. 

Check out his full commentary here.