« July 2010 | Main | September 2010 »

August 2010

Tuesday, August 31, 2010

Taming the Wild Algos

Posted by John Bates

"And now," cried Max, "let the wild rumpus start!"

— Maurice Sendak: Where the Wild Things Are


It’s not just equities and futures markets where strange stuff happens! An “algo gone wild” was spotted in the oil market (it actually happened earlier this year) and intrepid Reuters journalists got to the bottom of it.


High frequency trading firm Infinium Capital Management is at the center of a six-month probe by CME Group (and reportedly the CFTC) into why its brand new trading program malfunctioned and racked up a million-dollar loss in about a second, just before markets closed on Feb. 3. The algorithm, which was brand new, went live 4 minutes before the end of trading. It fired in 2000-3000 orders per second before being shut off. The oil price surged $1 then slid $5 over the course of the next two days. Read about the full story here:



I know the CEO of Infinium Chuck Whitman from the CFTC technology advisory committee – he’s a good guy and very knowledgeable. I believe him when he says his wild algos had no malicious intent – the algos were just broken and shouldn’t have been put live.


With algorithms and HFT comes the possibility of mistakes. Many more firms outside of the equities world are embracing HFT and their inexperience can cause market disruptions such as the Feb 3rd CME issue. A flash crash in oil or other commodities - or even foreign exchange - is not to be scoffed at. In fact, many commodities markets are much less liquid and homogenous than equities, and can be even more vulnerable to mistakes or manipulation. In the case of Infinium, the algo caused a spike in trading volumes by nearly eight times in less than a minute. It was a classic case of the algo running wild until it faltered and 'choked'. This is not how HFT strategies are supposed to work.


There are a number of best practices that can be used to mitigate against algos going wild:


The first best practice is diligent backtesting – using historic data and realistic simulation to ensure many possible scenarios have been accounted for. What does the algo do in a bull market, a bear market, at the open, at the close, when unexpected spikes occur, during a flash crash, when non-farm payrolls or other economic news is released etc. etc.? Of course there’s always the possibility of a “black swan” scenario – but then there’s always the possibility of an earthquake in London – but I bet the buildings aren’t built to withstand one – it’s a matter of covering likely possibilities as best you can. A backtesting process needs to be streamlined of course – as short time to market of new algos is key.


A second best practice is building a real-time risk firewall into your algo environment. Just like a network firewall stops anomalous network packets reaching your computer, so the risk firewall should stop anomalous trades getting to trading venues. These anomalous trades might be human or computer generated – such as “fat finger” errors, risk exposures (for a trader, a desk or an institution) being breached, or even algos gone wild (e.g. entering spurious loops and spitting out anomalous orders). Real-time risk monitoring is a second level protection for those problems you don’t catch in backtesting.


A third best practice is to use real-time market surveillance in your algo environment. Even if trades do not breach risk parameters, they may breach compliance rules, regulations or may be perceived by a regulator as market manipulation (by accident if not design). Detecting these patterns as they happen enables good internal policing by trading firms, rather than investigation or prosecution by regulators.


An algorithm is a tool in a trader's toolkit, and it needs to be taken care of as such. If it is well-oiled and the trader or quant or risk manager monitors its progress then the algo will do its job quickly and nicely. If the trader/quant/risk manager doesn’t properly prepare the algo or ignores the algo and lets it get rusty, so to speak, it could lose its edge and run amok. Algorithms must be monitored constantly for performance and for errors, and sometimes tweaked on-the-fly to ensure best results. A good algorithmic trading platform will enable trading firms to do just that.


Trading firms are not the only ones who need to be on guard for possible algos gone wild. In the case of Infinium, the regulators and the exchange were also slow on the uptake. This shows that everyone needs to be proactive in using the correct tools to monitor algorithmic trading. Sensing and responding to market patterns before the aberrations or errors have a chance to move prices is the right thing to do - in all asset classes. Be like Max and tame the wild things!

Thursday, August 26, 2010

The Risks of Being Caught Naked

Posted by John Bates

According to the FT, the Financial Industry Regulatory Association is undertaking a “sweep” of broker-dealers that offer direct market access to high-frequency traders to find out if they have pre-trade risk management controls in place for their algorithmic customers. (Read the full story here: http://tinyurl.com/28rg287). Here at Progress Software we have been advocating the use of pre-trade risk management tools for some time. The prospect of High Frequency Trading (HFT) firms running trades directly through broker algorithms (naked access) to exchanges/ECNs without adequate supervision has always concerned us. Brokerage firms simply give their clients an exchange ID number, which the clients then enter into their algorithms so that they can trade directly with the exchange.

Regulators are right to be concerned. Unfiltered access to trading destinations can end up causing trading errors or worse. Anything can happen with naked access; from fat fingered errors, trading limits being breached, even fraud - all of which can cost brokers, traders and regulators big money.

Although the SEC proposed banning naked access in January, and the Committee of European Securities Regulators (CESR) is likely to follow in its footsteps, there has been considerable pushback from brokers and trading firms. They say that adding a pre-trade risk management step would cause a "latency hop" in the trading process, claiming it would make them uncompetitive.  A layer of risk management often adds a third party into the mix, and - consequently - a delay.

But if everyone is required to add real-time, pre-trade risk management, then the competitive disadvantage is moot. The benefits of being able to pro-actively monitor trades before they hit an exchange or ECN far outweigh any microscopic latency hops in any event. The naked access playing field is hardly level. There are trading systems out there that claim to throughput a trade in 16 microseconds from outside an exchange's firewall, while exchange and ECN systems range anywhere from 250 to 700 microseconds (according to measurements by latency management firm Corvil).

Latency is clearly an important issue and complex event processing offers a solution. Brokers can achieve ultra-low-latency pre-trade risk management without compromising speed of access.  An option is a low latency "risk firewall" utilizing complex event processing as its core, which can be benchmarked in the low microseconds.  With a real-time risk solution in place, a message can enter through an order management system, be run through the risk hurdles and checks, and leave for the exchange a few microseconds later.

It is the ideal solution to a tricky question - how do I manage risk before my trade hits the exchange without compromising speed? The benefits are clear - a latency hop of a few microseconds not only saves money, it can also prepare your firm for the regulatory future.

Monday, August 23, 2010

Evacuate the Dancefloor

Posted by John Bates

Looking for all the world like someone yelled "fire" in a crowded nightclub, prop and quant traders are stampeding out of investment banks and headed for the hedge fund world. Some, mainly the prop traders, are being pushed gently out the door as banks prepare for the Volcker Rule (http://tinyurl.com/39ap28d). Others, like the quants (http://tinyurl.com/23c5h6d), are in search of the mega-bonuses that their prop trader or hedge fund manager compatriots are (or were) getting.


Impending changes in regulation are prompting banks to spin off proprietary trading activities, many by expanding their operations overseas where Messieurs Dodd and Frank cannot reach them. I’m very concerned about this “regulatory arbitrage” in which firms may move away from the US to find less strict regulatory regions. We don’t want to lose the lead in this important area of the economy.


Spin offs and regulatory arbitrage may well leave a herd of US traders looking for work and many may end up working at - or starting - hedge funds. The quants, having slaved over hot computers for the last few years to line bankers' pockets, are forming their own trading companies or joining prop trading firms with a profit-sharing deal.


Most of these traders will be in for a rude awakening when they sit down to work. Prop traders joining hedge funds will find that the technology budgets may not be as generous as they were at their last bulge bracket employer's firm. The quants, who are essentially programmers, will face huge challenges in finding firms that have the kind of low latency, scalable architecture that they need to design, tweak and trade with their algorithms. The level of trading freedom is different, too. Hedge fund managers will have something to say about a trader's profits - or lack thereof. Quants may find that designing an algorithm and handing it over to the trading desk is not quite the same as being responsible for the profits that the algo makes - or doesn't make.


Make no mistake, these prop traders and quants are highly intelligent and adaptable people. There will be many challenges to face going forward, but technology need not be one of them. There are instantly useable, scalable platforms that quants and hedge funds can use to build and deploy algorithms. These platforms, such as Progress Apama's Complex Event Processing Platform, offer a robust technology infrastructure to successfully create, test, deploy and manage their algorithmic strategies.


Algorithmic trading software is constantly transforming. As the volume of real-time market data continues to increase, algorithmic trading solutions demand an infrastructure that can respond to market data with near zero latency. To trade effectively in competitive markets requires rapid, opportunistic response to changing market conditions before one's competition can seize those opportunities. The people that are running for the doors and into the arms of hedge funds or other trading firms, will need this advantage. Competition is fierce, and their previous employers already have the technology advantage.

Thursday, August 12, 2010

The Busy "B" in BRICS

Posted by John Bates

Here in the United States algorithmic and high frequency trading have been under fire lately, blamed for everything from the May 6th flash crash to ruining small investors' profit margins. In the rest of the world, however, algorithmic trading is being recognized as both an opportunity and a way to attract liquidity and it is starting to blossom.


Exchanges in several emerging nations are getting with the electronic trading program - many by doing cross-listing (and even cross-ownership) deals with some of the larger, more automated exchanges. As Jeremy Grant put it in an article for the FT: "You scratch my back, and I’ll scratch yours." (see link here: http://www.ft.com/cms/s/0/123ece18-99ad-11df-a852-00144feab49a.html). Eurex and the CME have done many such deals recently, spanning Brazil, Malaysia, India, South Korea and Mexico. It makes sense for both parties - the big exchanges get exposure to new markets, and the smaller ones get to take advantage of the electronic expertise of the larger ones.


Brazil was one of the first to get on board when Brazilian Securities, Commodities and Futures Exchange (BM&FBovespa) was created by the merger of Brazil’s derivatives and stock exchanges in May 2008. Of all the BRICs (Brazil, Russia, India and China) Brazil is the real busy bee. In June 2009 the BM&F closed its “open outcry” pits in favor of an all-electronic GTS trading system (already introduced by Bovespa in 2004). The same month, the exchange joined the global trend to “co-location”, allowing customers to install servers inside the exchange’s data centers, ensuring low-latency for high-speed automated trades. These two moves set up BM&FBovespa as a destination that attracted algorithmic trading like bees to honey. The big banks are flocking in - Bank of America, UBS and Goldman Sachs have already announced the availability of algorithmic trading in Brazil. And an avalanche of brokerages have geared up with technology to enable algorithmic trading for their customers.


Brazil’s appetite for automated trading – particularly high-speed, low-latency algorithmic trading – remains very keen. Progress Software is excited to be part of this growth story. In February 2009, Ágora Corretora, one of Brazil’s largest brokerages, offered custom algorithmic trading strategies to its buy-side clients, through Progress Software’s Apama algorithmic trading platform. Around the same time another Brazilian brokerage, Alpes Corretora, deployed the same Apama algorithmic trading to develop and deploy execution arbitrage strategies for futures and equities.


Since mid 2008, over twenty Brazilian banks and brokerages rushed to make the switch from manual to electronic trading using Progress Apama’s algorithmic trading platform. These also include FinaBank Corretora, Banco Fator and most recently Link Investmentos, which bought in Apama’s algorithmic trading platform in March 2010. And many of these large banks are now offering their buy-side clients the chance to fine-tune their already customized algorithmic trading strategies, as per in Europe and the US. (It’s worth noting Apama also has direct buy-side clients.) Our Brazil success story has much to do with the fact that Progress is still the only capital markets-focused technology provider with an established local presence in São Paulo, Brazil, offering customizable technology.


BM&FBovespa will doubtlessly continue to incentivize both local institutions and international investors. The exchange’s tie-ups with CME and Nasdaq OMX Group will continue to drive new business growth in Brazil this year and into 2011. And the introduction of co-location at the Bovespa will further boost algorithmic trading both locally and internationally.


Brazil has been an incredibly exciting success story for Progress and our customers over the last 2 years. Now other firms are beginning to sit up and take notice of the region! But Progress was there first; we’ve gathered huge experience in the region and we remain ready and willing to provide more new customers with the ability to deploy and customize algorithmic trading solutions.


So Brazil is busy ‘B’! I’m looking forward to reporting more on the RICs in BRICs soon.

Wednesday, August 04, 2010

Algorithmic Terrorism

Posted by John Bates

At the CFTC's first Technology Advisory Council meeting on July 14, there was concern expressed around the concept of quote-stuffing. There was some evidence presented that the May 6th flash crash may have been caused by or exacerbated by this activity. While with regard to the flashcrash, other market experts I’ve spoken to know dispute this was the cause, quote-stuffing is a topic worthy of discussion


At the CFTC meeting, where I was an invited participant, data was presented from trade database development firm Nanex, which suggested quote stuffing contributed to the destabilization on May 6th. In this case the data suggests huge numbers of quotes were fired into the market on particular symbols (as many as 5000 per second) and that many of these were outside the national best bid/offer (NBBO). So what’s the point of this? Well with latency as a key weapon, one possibility is that the generating traders can ignore these quotes while the rest of the market has to process and respond to them – giving an advantage to the initiator. Even more cynically one can consider these quotes misleading or even destabilizing the market. In fact, Nanex state in their paper: "What we discovered was a manipulative device with destabilizing effect". Quote stuffing may be innocent or an honest mistake, but Nanex's graphs tell a very interesting tale (http://www.nanex.net/FlashCrash/CCircleDay.html). There are patterns detected - on a regular basis - that one could conclude is quote stuffing for the purpose of market manipulation. There's a very good article by Alexis Madrigal that discusses the research and issues in more detail (http://www.theatlantic.com/science/archive/2010/08/market-data-firm-spots-the-tracks-of-bizarre-robot-traders/60829/).


At the extreme, quote-stuffing could operate like a “denial of service attack” – firing so many orders that the market can’t cope - and crippling the trading of certain symbols, certain exchanges or the whole market. An influx of orders in sudden bursts to one exchange on one stock can slow down that system as it tries to process these orders. Nanex notes that there are 4,000 stocks listed on the NYSE and nine other reporting exchanges in the U.S. If each reporting exchange for each stock quoted at 5,000 quotes per second it would equal 180.0 million quotes per second. A daunting task no matter how advanced their processing technology is.


Without trying to overstate the issue, in the most extreme circumstances these practices could be considered algorithmic terrorism. One can imagine how, at the extreme, it is potentially catastrophic. The concern is that a well-funded terrorist organization might use such tactics in the future to manipulate or cripple the market. So much of our economy is underpinned by electronic trading – so protecting the market is more important than guarding Fort Knox! Regulators, such as the CFTC and SEC are taking this seriously - and need to respond.