Predictions for the Hybrid Data Landscape

The Age of Data has arrived, with new data sources, targets and processing models proliferating madly across enterprises of all sizes. While data has never been more valuable to a business — it now informs the who, what, where, when and how of decision-making — this new hybrid data landscape introduces new challenges. We anticipate the following innovative efforts in data management, integration and analytics to address these challenges.

The Rise of HTAP – Best of Both Worlds in Data Management

One of the most exciting trends for the balance of this decade will be HTAP (Hybrid Transactional/Analytical Processing), which is a Gartner-coined term representing a hybrid, converged software infrastructure that can handle both traditional transactional data management workloads AND modern analytic data management workloads.

Every business is struggling to find tools and techniques to effectively analyze the volume, variety and velocity of data. A new generation of columnar analytic SQL databases (like Actian Vector) will be critical to delivering on the promise of data-driven decisions. At the same time, organizations are familiar with, and trying to preserve, their investment in traditional transactional SQL databases (like Actian Ingres) that represent the backbone of data management in most organizations. How to marry those two data management needs?

What if you could have both capabilities in the same database? What if you could have the best of both worlds? Robust, enterprise-class OLTP database capabilities that leverage a 30+ year history of pioneering work in data management. And then add the world’s highest-performance columnar analytic database engine (with vector processing) into the same database infrastructure. One database, one security model, one SQL, one vendor – providing an innovative hybrid of operational and analytic processing that covers the entire spectrum of data management! With the ability to deploy to the cloud or on-premise. Now that is something to get excited about.

The Rise of Edge Databases for IoT Data Management
The emerging IoT stacks and solutions are missing one important element of scalable architectures – an elastic middle tier that can sit at the “edge” of the network and deliver robust processing services to the onboarding and analysis of IoT data. Most conventional IoT architectures focus simply on the two main end-points – the sensors themselves, spitting out low-level data, and the cloud, where sensor events should eventually “land” for analysis.

The sheer volume and repetition of sensor data make it impractical to imagine “landing” all sensor data in the cloud. The smarter IoT architectures will provide an intelligent middle tier – a kind of gateway function that resides near the sensors, at the edge. This layer is intended for early capture, processing and local analysis of the sensor data before only vital information is sent to the cloud.

The natural technology to deploy at the onboarding “edge” of the network is a bullet-proof embedded IoT edge database. Apart from the obvious advantages of deploying an embedded IoTDB at the “edge” of the network (persistence, security, etc.), you could also apply crucial local filtering (e.g. duplicates, errors, steady states, etc.) and data operations (e.g. sorts, aggregates, model application and local analytics) on the data prior to “landing” the data in the cloud – a much more efficient and productive setup for cloud-based analytics of sensor data.

The Rise of Hybrid Integration Platforms
It seems that regardless of how much we invest, integration remains an unsolved problem – permanently atop the priority list in all IT shops and organizations. The diversity of IT systems guarantees a baseline of integration challenges. An uncountable number of new end-points every year exacerbates the situation. Factor in that old and new end-points are changing constantly, and you multiply the problem further. Add the requirement for different integration patterns and delivery models and you begin to see the many intimidating dimensions of the integration problem.

Is there hope? Yes, tools that surpass the limited nature of today’s typical integration offerings are making their way into the market. Instead of focusing on one dimension of today’s integration problem – legacy on-premises ETL, heavy EAI tooling or lightweight cloud services, we will see customers turn to hybrid integration platforms – modern, dynamic and cloud-based solutions – to tackle all dimensions. Whether it is the variety of end-points (cloud, mobile or on-prem), or the variety of patterns (A2A via APIs or B2B via data), or the variety of skills (IT expert to LoB practitioner) or the variety of delivery models (cloud or on-premise), a modern hybrid integration platform like the Actian DataCloud will enable customers to adapt to today’s data integration needs.

The Rise of Graph Analytics in the Cloud
Neo4J, the leading commercial provider of on-premises graph database technology, recently raised a funding round of $36 million. This funding establishes graph databases (and the associated graph analytics space) as first class citizens in the pantheon of modern analytic techniques.

Why graph? In the now-immortal words of Donald Rumsfeld, there are “known knowns” (handled via BI and reporting), there are “known unknowns” (handled via predictive analytics to get a grip on a known analytic challenge such as fraud), and then there are “unknown unknowns.” These are the questions you never knew to ask, the queries you never knew to write. What are the unknown/unseen patterns hidden away in your data, and how do you find them? This is one of the great analytic challenges in datasets – what are the inherent (but unseen) relationships in the data – what objects are “close” to what other objects? What objects are “outliers”? What heretofore seemingly unrelated events share space and time?

It is exactly for this reason that graph is an important new analytic weapon. Graph analytics in the cloud are the ideal implementation platform, and we expect to see offerings that let you transfer your data into the cloud, load it into a back-end graph datastore like Actian Versant, and then “graph it” to see patterns inherent in the data (and even see new patterns emerge spontaneously as you add more data).

About Actian Corporation

Actian makes data easy. We deliver cloud, hybrid, and on-premises data solutions that simplify how people connect, manage, and analyze data. We transform business by enabling customers to make confident, data-driven decisions that accelerate their organization’s growth. Our data platform integrates seamlessly, performs reliably, and delivers at industry-leading speeds.