Event Processing blog

A foray into Beijing

Beijing was the last stop on my three city Asian tour and, from a personal perspective, the most exciting one as I’d never visited mainland China before.

China’s seemingly inexorable economic rise has been well documented. In the last 20 years, China’s GDP growth has averaged over 9%. As I travelled from the airport into Beijing’s central business district I saw few older buildings. Virtually everything, including the roads, looked as if it had been built in the last 10 years.

The Chinese stock market is big. In terms of the total number of dollars traded, the combined size of the two stock exchanges, Shanghai and Shenzhen, is approximately double that traded in the next biggest Asian market, Japan. The increase in stock trading has been very rapid. Trading volumes on Shanghai and Shenzhen have risen by approximately 20 fold in the past 5 years, although there has been significant volatility in this rise. 

The domestic algorithmic trading market is nascent. Currently, intra-day trading in company shares is not allowed. It is the recently established futures markets therefore where algorithmic and high-frequency trading are taking place. No figures exist on the proportion of trading done algorithmically in China currently, but I’m going to estimate it at 5%.

I was in Beijing to participate in the first capital markets event Progress has held there. Although Shanghai is the finance capital of China, we chose to hold the event in Beijing to follow up on previous work we'd done there. In the end, we had about 60 people along from domestic sell-side and buy-side firms attending which was a great result considering the relatively low profile Progress has at present in this market. There was optimism and an expectation that algorithmic trading had a bright future in China. 

I believe it's a practical certainty that the Chinese market will adopt algorithmic and high frequency trading. In every developed market a high, or very high, proportion of trading is done algorithmically and, although different regulations and dynamics make each market unique, nothing except an outright ban will prevent widespread adoption in every market in time. Liberalisation in China is occurring. For example, stock index futures are now traded, exchanges are supporting FIX, short-selling has been trialled and it is now easier for Chinese investors to access foreign markets. Also, earlier this year, the Brazilian exchange, BM&FBovespa, and the Shanghai exchange signed an agreement which may result in company cross listings. Only some of these changes support electronic trading growth directly but all are evidence that the liberalisation necessary to support such growth is happening. Inhibitors remain: no intra-day stock trading, restrictions on foreign firms trading on Chinese markets thus preventing competition and knowledge transfer from developed markets, and tight controls on trading in Renminbi. The Chinese regulators will continue to move cautiously. 

The question is not if, but when. We expect to sign our first Chinese customers soon. China is becoming a very important blip on the radar. 

 

Only You Can Prevent Flash Crashes

Faster than a forest fire, the May 6th, 2010 flash crash was a breathtaking nearly-1000-point drop-then-surge on the Dow Jones Industrial Average that shocked traders and the general public to their core. Unlike Smokey the Bear, who can sniff out and fight fires before they start to flame, US regulators only became aware of the flash crash after it was over. The flash crash left scorch marks that have scarred the reputation of the bulwark U.S. stock market and singed investor confidence.

Regulators were completely stumped as to the cause. Months of investigations unearthed one very important piece of information: the regulators were unprepared and ill-equipped to deal with this kind of event. They had no method of sniffing out the smoke that could lead to a flash crash fire.

A year ago, the flash crash revealed the first significant sign of growing pains in a new generation of trading. This new generation introduced the capacity to let computer models do the dealing; it allowed clever traders with clever programmers to build complex algorithms that buy and sell in the blink of an eye. All of this progress was made with little thought to the possible repercussions of a mistake.

Thus it was that on May 6th, 2010 when a mutual fund in Kansas entered a rather large ($4.1bn) sell order in E-mini S&P 500 futures contracts on the CME, the reverberations were felt throughout the marketplace. The order sparked a totally human panic on a day when fear was in the air and sentiment was leaning toward the bearish. The fire was then fanned by algorithmic trading strategies and HFT, causing an unprecedented drop within minutes and wiping out $1 trillion in market value before recovering. That a simple mistake could take the market down so fast was unthinkable.

Since the original flash crash we have seen dozens of smaller ones, which barely warranted a thimbleful of ink in the press:

• A mini flash crash nearly wiped out ten ETFs on Nasdaq OMX and NYSE Euronext on March 31st. The exchanges (quicker off the mark this time than on the May 6th flash crash) cancelled some of the trades in 10 out of 15 Focus Morningstar ETFs that had just been launched. Some of them tumbled by 98% before circuit breakers kicked in. The culprit appeared to have been a human being with fat fingers;  ETF market maker Knight Capital Americas took the blame.
• Last September the CME Group plugged some new contracts into Globex to try them on for size and they mistakenly went live. They were almost instantly traded - for about six minutes. Futures brokers noticed oddly anomalous spread price activity during that period, when trading is usually slow, and CME pulled them off.
• In the oil market last year high frequency trading firm Infinium Capital Management's brand new trading program malfunctioned and racked up a million-dollar loss in about a second, The new algorithm went live four minutes just before markets closed on Feb. 3rd and fired off 2000-3000 orders per second before being shut off. It caused a spike in trading volumes by nearly eight times in less than a minute and the oil price surged $1 then slid $5 over the course of the next two days.

And as energy and other asset classes outside equities - commodities, FX, derivatives - become increasingly automated there will be more flash crashes. Increased interdependence of asset classes will lead to cross asset flash crashes – a domino effect where the crashes 'splash' across asset classes, possibly wreaking havoc for market participants and regulators.  As regulators said following the flash crash: "a complex web of traders and trading strategies"  links the fragmented multitude of markets here in the U.S.    

Band-aids have been applied by the SEC; circuit breakers that prevent further trading when a stock moves more than 10% during a five-minute period. Market makers have been strong-armed to ensure they don't disappear when the market wobbles. The joint CFTC-SEC Advisory Committee on Emerging Regulatory Issues has made 14  recommendations on the May 6 Flash Crash including: implementing a limit-up/limit-down system, more extensive market access rules, studying the impact of maker-taker pricing and cancellation rates, and looking at a more cost-efficient audit trail.

These are a good start, but far from address the underlying problem. Like Smokey the Bear, regulators need a way to sense the smoke before the fire takes hold of the marketplace. Luckily there exist responsive and intelligent algorithms that can sense and react instantaneously to market anomalies and anticipate interruptions to liquidity. These rapid response algorithms could help to prevent the next flash crash by alerting risk and compliance managers of impending issues, or by changing trading strategies to accommodate market glitches. They can smell the virtual smoke and help to put out the fire before it starts.

On top of smarter algos, there are a few other splash crash prevention measures:

• Diligent backtesting – using historic data and realistic simulation to ensure many possible scenarios have been accounted for. A backtesting process needs to be streamlined of course – as short time to market of new algos is key. 
• Real-time risk monitoring - building a real-time risk firewall into your algo environment. Just like a network firewall stops anomalous network packets reaching your computer, so a risk firewall should stop anomalous trades getting to trading venues. 
• Real-time market surveillance. Even if trades do not breach risk parameters, they may breach compliance rules, regulations or may be perceived by a regulator as market manipulation.

Everyone needs to be proactive in using the correct tools to monitor algorithmic trading. Sensing and responding to market patterns before the aberrations or errors have a chance to move prices is the right thing to do - in all asset classes. Brokers need better pre-trade risk controls to prevent fat fingered trades from getting to market. Algorithmic trading means that the market moves too fast for exchanges or other venues to detect problems before they impact prices.  The detection of abusive patterns must happen in real-time, before any suspicious behaviour has a chance to move the market. This approach should be taken on board not just by the regulators, but by the industry as a whole. Only you can prevent flash crashes. 

 

The Trouble with Algorithms: Wild Children or Reckless Parents?

Algorithms and high frequency trading have been blamed for everything from the credit crisis to the May 6th flash crash and high speed market abuse, and have attracted unwanted interest from regulators on both sides of the pond. But questions remain whether these tools are really computer models gone wild or whether they are the spoiled children of reckless parents - regulation.

According to Dictionary.com, the definition of reckless is to be utterly unconcerned about the consequences of an action. One could argue that the Regulation National Market System was designed without regard to some of the consequences down the line. Blaming the wild children, algorithms, is to ignore that the parents - RegNMS - were somewhat reckless in designing the system.

In a blog on the TABB Forum on January 24^th,  Steve Wunsch of Wunsch Auction Associates explained that the system was working the way it had been designed.

"What really went wrong in the stock market on May 6? Prices aside, all of the plumbing was working fine. Not only were there no fat fingers, rogue algos, manipulators or terrorists at work, there were no significant breakdowns of order routing systems or data systems or any other elements of the stock trading infrastructure," wrote Wunsch.

Meanwhile, the National Commission on the Causes of the Financial and Economic Crisis in the United States released its report (Jan. 27th) and HFT was not mentioned at all. Nor were algorithms, as such, but 'computer models' were vindicated. The report said: "The crisis was the result of human action and inaction, not of Mother Nature or computer models gone haywire."

And it criticized regulators for not doing their jobs: “Widespread failures in financial regulation and supervision proved devastating to the stability of the nation’s financial markets.”

The result of the credit crisis and market meltdown in Sept. 2008 was the Dodd-Frank Act, which attempts to prevent another Sept. 2008.  But the flash crash insinuated itself into the picture, pointing out that no one had baked that possibility into the market reforms.  And, ironically, the market reforms set the stage for more flash crashes.

At the Tabb Forum Derivatives Reform Event a couple of weeks ago, a lot of people commented that Dodd-Frank puts in place a market structure that injects the equities and futures markets model, along with fragmentation, price transparency, streaming  quotes, into other asset classes. This theoretically invites algorithmic and high frequency trading and the threat of more flash crashes. At the event Peter Fisher of BlackRock said that what keeps him up at night is a flash crash in the interest rate market, citing the market structure argument, but specifically pointed out that this possibility was not envisioned in Dodd-Frank. 

With more and more asset classes becoming tradable electronically, partly thanks to mandated swap execution facilities (SEFs), the possibility of truly wild or rogue algos and market abuse becomes increasingly inevitable. And, as we pointed out last week, the very real possibility of a flash crash splashing across asset classes - we call it a "Splash Crash" - rears its ugly head.

Although the evidence against algos gone wild is thus far anecdotal for the most part, the belief that they can and will go wrong permeates the industry. Market abuse such as insider trading and manipulation are undoubtedly more prevalent. Fat finger errors are easier to prove, and are a fact of life in a high speed, high stress electronic marketplace.

Stay Calm and Remain Vigilant

The antonym of recklessness is vigilance. The regulatory parents must be more vigilant when it comes to their arguably brighter and naughtier children - algorithms and HFT. With algorithms and HFT come the possibility of mistakes and abuse. Many more firms outside of the equities world are embracing HFT and their inexperience can cause market disruptions. A flash crash in oil or other commodities - or even foreign exchange - is not to be scoffed at. In fact, many commodities markets are much less liquid and homogenous than equities, and can be even more vulnerable to mistakes or manipulation.

 There are a number of best practices that can be used to mitigate against algos going wild:

• Diligent backtesting – using historic data and realistic simulation to ensure many possible scenarios have been accounted for. A backtesting process needs to be streamlined of course – as short time to market of new algos is key.  
• Real-time risk monitoring - building a real-time “risk firewall” into your algo environment. Just like a network firewall stops anomalous network packets reaching your computer, so a risk firewall should stop anomalous trades getting to trading venues.
• Real-time market surveillance. Even if trades do not breach risk parameters, they may breach compliance rules, regulations or may be perceived by a regulator as market manipulation.

An algorithm is a tool in a trader's toolkit, not a naughty wild child. If the regulator parents are vigilant, and algos are subject to practical controls and monitored constantly for performance and for errors, market participants can sense and respond to market patterns before the aberrations or errors have a chance to move prices.

-Dan

Beware the Splash Crash

We have had the flash crash, the breathtaking 1000-point drop-then-surge that happened on May 6th, 2010. In the near future we will have a new worry - prepare for the “Splash Crash”, which will cross asset barriers in a single bound.

As asset classes outside equities - energy, commodities, FX, derivatives - become increasingly automated there will be more flash crashes. Increased interdependence of asset classes will lead to cross asset flash crashes – a domino effect where the crashes 'splash' across asset classes, possibly wreaking havoc for market participants and regulators.  

As regulators said following the flash crash: "a complex web of traders and trading strategies" links the fragmented multitude of markets here in the U.S.  And, like dominoes, when one goes the rest follow. The dominos are no longer limited to one asset class. Algorithms are becoming increasingly sophisticated, encompassing all of the elements that may impact a trade in a certain instrument. If a trader wants to take a substantial position in a foreign equity, for example, there are many ingredients that can affect its market price.

Consider news events such as the BP oil spill or the current political crises in Egypt and Tunisia. The impact of these events has illustrated the close relationship between the oil price, equities, foreign exchange, commodity futures and the bond markets. Extreme and possibly unexpected events coinciding can trigger a cascade. We saw with the flash crash how instability in European economies caused nervousness in the market and then an algorithm did something unexpected – causing a cascading effect across futures and equities markets. As the cross-dependencies grow and algorithms become more inter-twined, so the risks for a “splash crash” grow.

It’s not hard to consider a splash crash scenario given the growing inter-linking of markets. For oil companies, such as BP, equity trader’s positions can be affected by the price of the pound and UK interest rates, as well as the dollar and any countries currencies where it is exploring or supplying oil. Oil prices will impact the bottom line, therefore the share price. Political events such as wars in oil-producing countries are important, as are disruptive events such as oil spills, and can impact oil, equity, bond, commodity future and foreign exchange prices. What if China's economy falters and its oil consumption is predicted to fall? Oil prices globally fall, dragging the US dollar higher. Oil derivatives predicting that prices would stay high for the next year or two or three also fall. Share prices for the whole oil sector including BP, Exxon Mobil, Texaco, Chevron, Philips, Sinochem, etc. collapse. Debt markets are stunned and bond rates rally.

All of these factors can be programmed into an algorithm that monitors and makes trading decisions on the BP position. If large banks and hedge funds also have substantial positions in BP, the dollar, the debt and the derivatives and they also have algorithms that will kick in when certain parameters are met. If enough instability and unexpected conditions occur and then one of these algorithms does something strange or unexpected, the cascading impact could be enormous across all asset classes. For example, massive automated sell orders for oil shares, energy futures and derivatives and buy orders for USD and Treasuries. Trading systems could clog up, limited bandwidth could choke orders, exchanges could freeze up - splashing across all of the affected asset classes. Pandemonium.

Splash Crash Prevention Tips

Luckily there exist ever more responsive and intelligent algorithms that can react instantaneously to market anomalies and anticipate interruptions to liquidity. These rapid response algorithms could help to prevent the next flash crash by alerting risk managers of impending issues, or by changing trading strategies to accommodate market glitches.

On top of smarter algos, there are a few other splash crash prevention measures:

• Use real-time pre-trade analytics and risk management. If the mutual fund in question on May 6th or its executing broker had done a thorough back-test of its trading strategy, using some of the dire indicators present, it might have thought twice about selling so aggressively - possibly preventing the crash.
• "Light up" the algorithmic trading process. Visibility during the trading process is crucial. Surveillance technology exists than can monitor the markets for anomalous behaviour and alert the parties involved if it is spotted. Give the regulators the tools, too. 
• Homogenize trading rules across all exchanges and ECNs. When one halts trading they all halt - for the same amount of time.

 

A postcard to Jeremy Grant

Jeremy Grant, editor of FT Trading Room at the Financial Times, recently asked for explanations "on a postcard" about why speed is a force for good in financial markets, or put another way, to explain what the benefits are of high frequency trading. I've just come back from Mexico where I was addressing the Association of Mexican Brokers and during my visit I thought I'd write that postcard. So here it is:

 

Dear Jeremy

I saw your request for postcards recently, and as I'm travelling I thought I'd drop you one. There's not a lot I like doing more than explaining the benefits of so-called "high frequency trading".

I would suggest that you think of high frequency trading, or HFT, as being just the latest stage in the evolution of electronic trading. And this, as you know, has evolved very rapidly over the last decade because of cheaper and faster computers and networks. It's led to many innovations and benefits: electronic crossing networks, algorithmic trading, online retail trading, smaller order sizes, the overall increase in trading volume, more price transparency, greater trader productivity, more accessible liquidity, spreads between buy and sell prices tightening, broker commissions reducing, competition between exchanges and so smaller exchange fees - none of these things would have happened without electronic trading. MiFiD couldn't have happened; it simply wouldn't have been financially viable for the many alternative European equity-trading venues to launch without cheap access to networks and computers. Without these we would still have greedy, monopolistic exchanges with high transaction prices.

HFT is just the latest step in a technology driven evolution. You can't just look at it in isolation.

"Ah", you exclaim, "but high frequency trading is a step too far. Trades happening far faster than the blink of an eye. Surely that can't be right?"

So what if trades happen quickly? Things "going too fast" is a common concern. In 19th century Britain, people were worried about trains going faster than 30mph. They thought that passengers would suffocate or that as the train reached a corner it would simply come off the rails! And to those that say trading happens too quickly, at what speed should it occur? If not micro or milliseconds, should it be a second, a minute, an hour? Who's going to decide? Any choice is entirely arbitrary anyway; time is infinitely divisible.

There are plenty of things that happen too fast for humans to comprehend - human nerve impulses travel at more than 100m per second, yet we function successfully. Why? Because we have the monitoring systems in place that ensure the information from the nerves is processed correctly. Put a finger on a hot coal and it will be retracted immediately - quicker than we can consciously think. And if a 200mph train goes through a red light then warning bells will ring and the train will be automatically stopped.

And so to the main point. Trading speed, per se, is not the problem. But, yes, problems there are. Markets, particularly in Europe and the US are now very complex. These markets are fast moving, multi-exchange, with different, but closely interlinked asset classes. It is this complexity we find difficult to understand. Speed is only one facet of this. We imagine that an armageddon incident could occur because we know that the markets are not being monitored properly. Regulators freely admit this - Mary Schapiro recently said that the SEC was up to two decades behind in its use of technology to monitor markets. And because we know that the people in charge don't know what's going on, we get scared.

It doesn't have to be like this. The same technological advances that led to the evolution of HFT can be used to ensure that the markets work safely, by ensuring that limits are not exceeded, that an algorithm "going crazy" can't bring down an exchange, that a drunken trader can't move the oil price and that traders are dissuaded from intentionally trying to abuse the markets.

Doing things faster is a human instinct. Faster, higher, stronger. The jet engine, the TGV, the motorway. Would we really go back to a world without these?

Breaking the Machines

(This piece originally appeared on TabbForum - linked here)

 

The furore over high frequency trading and rogue algorithms is turning these important trading tools into fodder for the mainstream media. As part of the Commodity Futures Trading Commission's Technology Advisory Committee (TAC), I had high hopes that my colleagues and I would be an instrumental part of solving some of these issues that very publically dog our markets, including mechanisms for detecting or even preventing another flashcrash, stopping algorithms going out of control and curbing rogue traders. I believe the vanguard of such market improvements is not draconian restrictions on trading algorithms but rather regulator-led best practices and market policing. If the right measures are implemented and suitably publicized, it would address the market’s nervousness and have algorithms smelling fragrant once more.

 

I was thus a little disheartened by the second TAC meeting last week (Technology: Achieving the Statutory Goals and Regulatory Objectives of the Dodd-Frank Act). Given the public fear that algorithms and high frequency trading are evil, I was concerned when one commissioner even went so far as to ask the question in his opening remarks as to whether algorithms should be banned completely. If this ever did happen in the US, heaven help our economy. I would equate such an action to the Luddites – a group in 19^th Century Britain that broke machines to protest against the industrial revolution. Algorithms are not evil; there are many positive aspects of algorithms and HFT. They minimize the market impact of large trades, lower the cost of execution, make more open and efficient markets, allow trading venues to evolve faster, encourage entrepreneurship and increase trader productivity, among many other things. Banning what is essentially the new industrial revolution, and now an integral part of electronic trading, could take us beyond a double dip recession and back into the dark ages.

 

A few key points came out in the flash crash report that really need to be emphasized. A key one is that there is a difference between algorithmic execution strategies and high frequency trading strategies. The former are manually set up and are designed to break up a large trade, typically executed in a broker on behalf of a buy-side customer. The latter are much more automated and continuously look for trading opportunities to act on, typically operated by a prop shop or hedge fund. The latter sounds scarier – but it was actually the former – or one particularly extreme instance of the former - that got the lion’s share of the blame in the joint SEC-CFTC report. HFT was pretty much exonerated. It was really human error in the way the execution algo that traded the E-mini was set up that was at fault. In fact one of the TAC participants actually made the point that many of the HFT algos had smarter monitoring built in – which made them pull back from the market when it started to go haywire. Yes that withdrew liquidity – but the HFT algos behaved sensibly given the circumstances.

 

Commissioner Scott O’Malia asked the question whether a rogue algorithm is the same as a rogue trader. Great question! An algorithm does not “decide” to go rogue, unlike a human rogue trader who is more deliberate. Usually a rogue algo is a mistake – such as Infinium’s algo that went wrong and fired thousands of orders into the market in February (http://www.reuters.com/article/idUSTRE67O2QQ20100825) or CME’s test algo that fired phantom orders into the market in September (http://www.ft.com/cms/s/0/706c45dc-c00a-11df-9628-00144feab49a.html). But rogue algos can threaten the well-being of a marketplace just as a rogue trader can. Rather than banning or restricting HFT and algos it would be much more productive to look at how they and the market can benefit from effective controls.

 

Some suggestions that I made on the TAC as to how we might provide more confidence around algo trading are as follows:

 

Firstly, market participants should be mandated to do better back-testing and algo monitoring to help prevent rogue algorithms and scoundrel traders from entering the market. Testing the execution algo that went wrong on May 6^th under realistic market conditions might have prevented it going live. More intelligent monitoring might have made it pull out of the market before it did deep damage. Real-time monitoring can detect and respond immediately to dangerous market conditions, “fat finger” or algo errors and trading risk exposures being exceeded.  As illustrated by the HFT algos that stepped out of the market on May 6th – some firms have better monitoring technology than others! The CFTC and SEC could provide best practices guidance and maybe even recommend data sets, simulators and pre-production processes to help with this.

 

Secondly, Exchanges should continue to enhance their monitoring and surveillance systems. Clearly we’ve not perfected it yet given that a rogue algo within the CME managed to fire in phantom orders as recently as September. Also, to ensure consistent response to market crises, all trading venues in a particular asset class should have consistent circuit breakers, which operate under the same circumstances. This would avoid some of the problems discussed in the flash crash report.

 

And the CFTC (as well as the SEC) needs to be "mission control" to monitor across all markets and provide an early warning system. If firms believe they can be watched in real time they will be much more careful. Unfortunately, the CFTC’s Chairman suggests that there is no budget for such technology and that they will have to rely solely on controls by the exchanges and trading venues. This is unacceptable. The importance of trading to our economy means that ensuring confidence in our markets combined with allowing the world’s most advanced forms of algo trading - with the necessary safety measures to prevent meltdowns - is a matter of national security! The regulators are the US Marshalls to HFT's Wild West. The CFTC should go to Congress and make the case for a bigger budget. And they should strike while the iron is hot.

 

The flash crash may have been a mixed blessing, having pointed out many market structure issues that the regulators should be striving to correct or control. Until that day American stock markets were the envy of the world, the model for modern trading -- fast, stable, efficient and for the most part transparent (http://tinyurl.com/29bpr4r). That perception has changed and the rest of the world is aiming to avoid, not mimic, our model. It is critical that the US take the necessary steps to remain the shining example of capital markets. The technology is there, it simply needs to be used. Most importantly we must not allow negative publicity to lead us into Luddite-style regulation and break the machines that are fuelling this new industrial revolution.

 

When Does a Rogue Become a Scoundrel?

Now that the Dodd-Frank Wall Street Reform Act is signed into law, there lies a mountain of work ahead for regulators. Making sense of the 2,000+ page document and turning it into viable recommendations and regulations will be an arduous process.

 

The Commodity Futures Trading Commission's second Technology Advisory Committee meeting titled “Technology: Achieving the Statutory Goals and Regulatory Objectives of the Dodd-Frank Act,” will be held on October 12, 2010 at 1:00 p.m., in Washington, D.C. (http://tinyurl.com/2vfdp4n). At the meeting, my committee colleagues and I will discuss some of these goals and objectives. Specifically, as a result of the SEC & CFTC's report on the May 6th flash crash, CFTC Commissioner Scott O'Malia has said that he wants to take a look at whether algorithms that cause disruption in markets - rogue algorithms - should be treated as if they were rogue traders.

 

Commissioner O’Malia said in the announcement of the October 12 meeting: “While I do not believe that the flash crash was the direct result of reckless misconduct in the futures market, I question what the CFTC could have done if the opposite were true. When does high frequency or algorithmic trading cross the line into being disruptive to our markets? And, along those same lines, who is responsible when technology goes awry? Do we treat rogue algorithms like rogue traders?"

 

This is an interesting topic. When does an algorithm 'go bad'? Is it the algorithm's fault? Of course not, an algorithm does not decide to go rogue. It is down to human error - either in the programming or the execution thereof. In the case of the flash crash a mutual fund chose a 'dumb' execution algorithm preset with inappropriate parameters to execute a large futures sell order in a market that was - by all accounts - ready to plummet. This circumstance illustrates how rogue algorithms can evolve as an unintended consequence of circumstance and/or human misjudgment.

 

When a trader goes rogue it is more deliberate. It can be because he is losing money and hiding it - as in the case of Jerome Kerviel at SocGen, or maybe he had too much to drink at lunchtime and was feeling invincible - like Steve Perkins at PVM. The former lost the bank over $6bn, the latter lost his brokerage $10m. These were very human errors, effectively the work of scoundrels.

 

What rogue traders and rogue algorithms have in common is that both can, in many circumstances, be detected early - or even prevented - through the use of better technology. Comprehensive pre-trade analysis, including backtesting algorithms under a wide range of circumstances, could have prevented the 'dumb' May 6th algo from having its way with the market. Thorough real-time risk management and monitoring could have spotted Kerviel's limit-busting trading patterns and his hiding the trades. Pre-trade risk controls would have kicked the PVM trader out of the system before he got in too deep.

 

It is no longer acceptable to blame rogues and scoundrels for market anomalies or for banks, brokers and buyside firms losing money. The technology is there, it simply needs to be used.

 

How One Very Naughty Algorithm Ruined Everyone's Day....

The chaos theory says that small differences in initial conditions can yield widely diverging outcomes. Thus it was that on May 6th when a mutual fund in Kansas entered a rather large ($4.1bn) sell order in E-mini S&P 500 futures contracts on the CME, the reverberations were felt throughout the marketplace. The order sparked a totally human panic on a day when fear was in the air and sentiment was leaning toward the bearish. The fire was then fueled and fanned by algorithmic trading strategies and high frequency trading, causing an unprecedented drop within minutes.

 

As regulators said following the flash crash: "a complex web of traders and trading strategies" links the fragmented multitude of markets here in the U.S. And, like dominoes, when one goes the rest follow. That a large sell order for E-mini S&P 500 contracts on the CME, executed by an aggressive broker's algorithm, had such a devastating knock-on effect throughout the whole of the marketplace is telling. At first, high frequency trading programs absorbed the liquidity but, as prices dove, they changed their tack and began to sell as well. Liquidity was choked off. As traders tried to make sense of the situation, it because clear that the rest of the market was a-tumbling.

 

The futures contract - technically an ETF - is a stock market index futures contract based on the S&P 500. And the S&P 500 after all is made up of 500 individual equities shares. Panic, although a human emotion, can also spread in the electronic sense. In this case when the ETF algos ‘panicked’, the 'emotion' spread across other asset classes in an instant. Inconsistent and inhomogeneous trading rules across the various destinations worsened the effect. As trading halted on NYSE other destinations kept churning, but liquidity was already strangled.

 

The post-crash joint report by the SEC and CFTC notes that on the morning of the flash crash there was a decidedly unsettled feeling in the market. The European debt crisis was top of the list, and risk indicators included higher premiums on credit default swaps for debt from Greece, Portugal, Spain, Italy, and Ireland and a weak Euro. The VIX shot up by over 30%, the fourth largest single-day increase. Prices on gold futures rose 2.5%, while yields of ten-year Treasuries fell nearly 5% as investors engaged in a “flight to quality", said the report.

 

Clearly it wasn't a good day to sell; but we all make mistakes. The report highlights just how easy it is to make a well-intentioned trading faux pas that can wipe hundreds of billions of dollars off the market within minutes.

 

Lessons have been learned and they will help to prevent another flash crash. I have some to add:

 

• Perform comprehensive pre-trade analysis, including by backtesting algorithms under a wide range of circumstances using realistic market simulation – If the mutual fund in question or its executing broker - had done a thorough back-test of its trading strategy, using some of the dire indicators and conditions present, it might have thought twice about selling so aggressively - possibly preventing the crash.
• "Light up" the algorithmic trading process. Visibility during the trading process is crucial. Market monitoring and surveillance technology exists than can monitor the markets for anomalous behavior and alert the parties involved if it is spotted. Red alerts should have been going off in the broker, with real-time risk analytics highlighting impending problems. Also, the regulators should have been able to see an “early warning” and respond from a NORAD-style monitoring HQ.
• Homogenize trading rules across all exchanges and ECNs. When one halts trading they all halt - for the same reason and for the same amount of time.

 

Gary Gensler, chairman of the CFTC, said following the report that perhaps both brokers and customers need to be obligated  "to monitor and make non-disruptive trading judgments." As Mr. Gensler noted, high volume is not necessarily an indicator of real liquidity. The more visibility we have into trading, the more responsive we can be and the more likely we are to avoid another flash crash.

 

No evil algo-trader behind the flash crash

The long anticipated joint SEC and CTFC report on the 6 May 2010 flash-crash came out last Friday.

After reading much of the report and commentary around it, I'm feeling rather underwhelmed.

The root cause of the flash-crash, the most talked about event in the markets this year, was a boring old "percentage-by-volume" execution algorithm used by a mutual fund to sell stock market index futures. How banal.

The algorithm itself was simple. It just took into account volume, not price, and it didn't time orders into the market. Many commentators have pejoratively described this algorithm as "dumb". It may be simple, but it's one of the most common ways that orders are worked - buy or sell a certain amount of an instrument as quickly as possible but only take a certain percentage of the volume available so the market isn't impacted too much. The problem was the scale. It was the third largest intra-day order in the E-mini future in the previous 12 months - worth $4.1Bn. The two previous big orders were worked taking into account price and time and executed over 5 hours. The flash-crash order took only 20 minutes to execute 75,000 lots.

It wasn't this order on its own of course. Fear in the markets created by the risk of Greece defaulting was already causing volatility. Stub quotes (extreme value quotes put in by market makers to fulfill their market making responsibilities) appear to have contributed. There was the inter-linking between the futures market and equity markets. There was the very rapid throwing around of orders - described as the "hot potato" effect, certainly exacerbated by the many high-frequency traders in the market. There was the lack of coordinated circuit breakers in the many US equity markets. There was the lack of any real-time monitoring of markets to help regulators identify issues quickly.

High-frequency and algorithmic trading have been vilified in many quarters over the last months. I think many were expecting that the flash-crash cause would be a malignant algo, designed by geeks working in a predatory and irresponsible hedge fund, wanting to speculate and make profits from "mom and pop" pension funds. It just wasn't anything of the kind.

The flash crash has raised important issues about the structure of multi-exchange markets, the role of market makers, the lack of real-time surveillance and how a simple execution strategy could precipitate such events. I do hope that the findings in the flash-crash report will ensure a more balanced view on the role of high-frequency and algo trading in the future.

Mystery Solved, but Questions Remain

Scooby Doo and the gang have been investigating the spooky mystery of the "phantom orders" on CME and they found out that the culprit was... wait for it.... The CME! Turns out the exchange plugged in some new contracts to try them on for size and they mistakenly went live and bam! they got traded.

According to the FT, CME Group said it had “inadvertently” posted test orders intended for its quality assurance procedure on Globex (http://tinyurl.com/23bpfjj). CME said the company tests its systems as a matter of course and it had not determined whether human error or a computer glitch caused the mistake.

The FT said the mistaken order flow began at 3:38pm ET time on Monday, September 13th and lasted for six minutes. Futures brokers noticed oddly anomalous spread price activity during that period, when trading is usually slow.

CME said it was working with the customers that somehow managed to trade these contacts in those six minutes, and says it won't "bust" the transactions. Fair enough. But the mistake shows how the exchange lacked adequate monitoring technology to see that some test contracts had entered the live environment - and were being traded.

This six minute phantom orders mystery is solved, but the very fact that it occurred supports my argument that brokers, traders and exchanges need to have more advanced capabilities to detect problems and abuse as they happen, and recommend actions to take in response. The Mystery Machine got to the bottom of the case, but it was not fitted with the latest real-time surveillance equipment. It seems, like Velma, the CME temporarily lost their glasses!