Hybrid Processing Enables New Use Cases – Business Impact Brief
The 451 Take
Organizations are increasingly turning to hybrid operational and analytical processing (HOAP) systems, which are capable of handling both types of workloads. In fact, our Total Data Market Monitor research suggests that these hybrid systems will rival transactional systems in terms of revenue for newly deployed applications. Adopters of hybrid systems will increasingly recognize the significant value they bring compared with single-purpose transactional and analytical systems.
Blending operational and analytical processing is not a new concept, although historically these have mainly been discrete processes. While there are good reasons for the processes to be separated, there are equally good reasons why they should be combined. Beyond eliminating the overhead of maintaining separate transactional and analytical systems, deploying hybrid databases enables organizations to run analytics on incoming operational data, taking advantage of the real-time ‘transaction window,’ which can be incredibly lucrative.
Many organizations are finding business success running hybrid workloads. Financial services firms, for example, are able to process transactions while simultaneously conducting analysis on these transactions to respond to end-user queries in milliseconds. Similarly, retail firms are able to embed analytics into business processes and events, and provide personalized offers to users in real time. Various IoT use cases can also benefit from processing sensor data and combining it with historical data for analysis in order to identify anomalies. This analysis enables organizations to move beyond preventive maintenance toward predictive maintenance, thereby improving equipment effectiveness.
Transactional systems (OLTP) and many analytical systems (OLAP) available today are based on the relational data model that was first introduced in the early 1970s. For the most part, this market has seen steady growth driven by maintenance revenue. However, zeroing in on incremental growth – i.e., growth from new applications, as opposed to recurring maintenance – it’s clear that hybrid systems will have a significant impact on database adoption choices in the immediate future.
Our Market Monitor data shows that hybrid workloads accounted for 10.1% of total incremental database revenue in 2016, compared with 59.6% for OLTP workloads, and 30.3% for OLAP workloads. By 2021, however, we estimate that hybrid workloads will account for 30.6% of incremental database revenue, compared with 39.3% for OLTP and 30.1% for OLAP. Growth in hybrid workloads will be driven by the value that organizations realize in performing intelligent analytics on transactions, which enables them to react in real-time.
COMBINING TWO SYSTEMS INTO ONE. Hybrid operational and analytical processing means having a single system, which leads to reduced time by eliminating ETL processing, decreased complexity associated with enabling real-time event-driven use cases, and not having to move data from one system to the other.
ENABLING NEW USE CASES. The primary use case for hybrid systems is real-time analytics on operational data that can be used to identify events from business applications before the data is transformed and loaded into a data warehouse/data mart. However, with advancements in database technologies and hardware, hybrid systems now offer more sophisticated analytical methods, particularly machine learning to support systems of engagement with systems of intelligence such as rules engines, decisioning systems, recommendation engines, natural language processing and image recognition.
CONCENTRATING ON HIGH-VALUE EVENTS. While hybrid systems can handle transactional or analytical workloads separately, organizations are likely to reap the greatest benefits when they match the right use case with true hybrid processing.
REDUCING COSTS. With fewer physical systems to maintain, hybrid systems may also require fewer in-house resources to maintain. In environments where transactional and analytical systems are separate, individuals often specialize in one or the other. With hybrid systems, individuals with skills in both transactional and analytical processing can be leveraged.
The future looks bright for organizations implementing hybrid systems. One significant reason is the ability to take action in real-time with regard to events and transactions. Another is cost reduction, with fewer systems to maintain. There have also been advancements in database technologies, particularly in distributed environments, and machine learning has played a key role for many organizations. This combination of new databases and advanced analytics represents a sort of perfect storm for enterprises to leverage hybrid systems across a variety of business processes.
While we anticipate broad applicability of hybrid processing based on our Total Data Market Monitor research, we believe that a handful of verticals will particularly benefit, including financial services, retail, healthcare, manufacturing, transportation/logistics and IoT.
Our research suggests that in order to take full advantage of hybrid processing, organizations will rely on a core set of capabilities. The first is the ability to process significant quantities of data and events in real time or near-real time. Organizations must also be capable of scaling to handle growth spikes as well as concurrency demands. And then there is the ability to accept a wide variety of data from multiple sources, including both existing systems and systems outside the organization.
Based on 451 Research Market Monitor data, we expect hybrid operational and analytical processing systems to see significant growth, particularly as organizations find ways to leverage hybrid processing for new business opportunities and to increase operational efficiencies.