Increasing trade volumes and periods of high market volatility are creating significant technology challenges for financial services firms engaged in electronic trading. This is especially true for sell-side firms, since they partition already large volumes of incoming orders into even more child orders for execution. At the same time, they must support real-time concurrent analytic queries to provide risk management, surveillance, order status, and other information for internal and external clients. This need for multi-workload processing at a massive scale with the most demanding requirements for performance and reliability has traditionally been difficult and expensive to satisfy.
Compounding the challenge, unexpected world events that affect markets can cause transaction volumes to spike dramatically. Recent examples include the 2008 financial crisis, the 2010 “Flash Crash,” and the devaluation of the Chinese currency in 2015.
A critical component of a sell-side firm’s technology infrastructure is its transaction management and analytics platform. The platform must be extremely reliable and highly available, able to withstand both normal transaction volume growth as well as the extreme spikes that can occur during periods of market volatility. A failure, or even just a slowdown of the transaction management and analytics platform, can create severe consequences. For example, it can take many hours to rebuild order state and resume normal operations after a failure, during which time the firm’s ability to process additional trades and provide order status is compromised and financial losses mount. Even a slight delay or outage can cause significant financial losses and impact the firm’s reputation. One major bank recently reported losses of $100,000 for each minute of system downtime.
In order to successfully handle growth and volatility without performance or availability issues, the platform must balance transactional workloads with the concurrent analytic demands of downstream applications at scale. Firms must be able to process millions of messages per second, while simultaneously supporting thousands of analytic queries from hundreds of systems that must report on the state of orders and perform other queries.
A hybrid transactional/analytical processing (HTAP) data platform can deliver the performance equal to or greater than an in-memory database with the persistence and reliability of a traditional operational database. Unlike other database architectures, HTAP data platforms are designed to accommodate both very heavy transactional workloads and a high volume of analytic queries – concurrently – on the transactional data. Reliable HTAP data platforms operate without incident or performance degradation even during periods of extreme market volatility.
One major sell-side investment bank is achieving a 3-5x throughput increase and a 10x performance increase with a 75% reduction in operating costs compared with their previous system, and has operated without incident since its inception in 2007.
Many financial institutions are in the process of evaluating the effectiveness and complexity of their legacy database technology stacks. Those planning a technology refresh for their transaction management system should consider investigating an HTAP data platform.
