451 Research Report: A New HOAP?
Hybrid Operational Analytic Processing and the Future of the Database Market
Data from 451 Research’s Total Data Market Monitor indicates that databases designed to support a combination of operational and analytical processing workloads will quickly become mainstream, at least for new application projects.
451 Research has previously identified the emergence of a new breed of database providers with products that are positioned for a combination of operational and analytical workloads, as well as the systems of intelligence workloads that they are used for. Data from 451 Research’s Total Data Market Monitor suggests that these databases that are designed to support hybrid operational and analytic processing (HOAP) will quickly become mainstream in the coming years – at least for new application projects.
The 451 Take
The blending of operational and analytical systems continues to add value for many organizations. And while hybrid systems may not be an ideal fit for every firm, there are many reasons they do make sense. Beyond the reduction in maintaining a separate transactional and analytical system, hybrid databases enable organizations to carry out analytics on incoming operational data, taking advantage of the ‘transaction window,’ which, if done right, could be incredibly lucrative. While most existing database applications do not take advantage of hybrid functionality, and will continue to account for the majority of database revenue for many years, we expect HOAP workloads to rapidly account for a significant proportion of incremental database revenue, and that supporting them will come to be expected in any mainstream operational database product or service.
It has become an accepted best practice over the past 40 years or so that analytics should be performed on data stored in a separate database from that used to support operational, transactional systems. While there are data management benefits to be gained from this approach, its origins lie not in architectural elegance but rather the need to avoid the performance limitations of traditional systems, which made it impossible to support high volumes of database reads and writes in the same environment.
For the most part, this remains true today – with databases tuned for online transaction processing (OLTP), the default choice for operational workloads and online analytic processing (OLAP), the default choice for analytic workloads. While most database products can be used to support both OLTP and OLAP workloads, they will typically be tuned to support one or the other, and for performance reasons, would rarely be used to support both at the same time.
However, over the past five years, we have seen the emergence of a new breed of relational and non-relational database vendors that claim to have improved performance by taking advantage of hardware, memory and processor functionality to an extent that allows them to support operational and analytical workloads in the same instance. The incumbent database vendors are also in on the act – adding in-memory columnar engines to their existing row-based databases in order to support hybrid workloads.
In recent years, multiple terms have arisen to describe these database workloads, the most popular of which is probably ‘hybrid transactional and analytical processing’ (HTAP), as coined by research and advisory company Gartner. It is often assumed that the term ‘transactional’ implies ACID (Atomic, Consistent, Isolated, Durable) transactional integrity, which would exclude most NoSQL non-relational databases for this category. While this is not necessarily the case, we prefer to use the term ‘operational databases’ to avoid confusion. Hence, our use of the acronym HOAP.