2019 Data Management Trends and How They Affect Integration

2019 is becoming an exciting year for the data management community. Many of the trends of the past few years have successfully made the transition from emerging concepts to mainstream solution approaches.

While trends are important building blocks of how companies approach their data management today, they are also providing insights into future capabilities to incorporate the individual pieces into a holistic, integrated solution.

The Cloud is Becoming the First Choice for New IT Systems

Companies have been moving applications and IT components to the cloud for a few years to save costs, improve performance and achieve solution scalability. During 2019, new developments are emerging in the cloud migration trend where most companies are looking to cloud solutions first for new IT systems and only considering on-premises solutions if the cloud isn’t feasible for some reason. This includes both ready-to-use SaaS solutions as well as cloud-based infrastructure (IaaS and Paas) for various needs, such as data warehouses and in-house developed applications. One of the latest cloud trends is integration-platform-as-a-service (IPaaS), which is a set of cloud-based capabilities for integrating both cloud-native and on-premises components for seamless interoperability.

Data Warehouse Migration to the Cloud

During the past few years, Hadoop has been the big trend in data warehousing. Many companies have implemented Hadoop on-premises and are now facing increasing operational costs as well as challenges to integrating with cloud-native solutions. The trend of 2019 is for companies to migrate data warehouses to the cloud where they can be operated at a lower cost and with greater performance and scalability. As this migration is occurring, many companies are examining their integration strategies and capabilities to evaluate how to bring more SaaS data sources and streaming data from IoT systems into the data warehouse for analysis.

Data Lake Reboot

The unstructured nature of data lakes made them all the rage during the past 5 years as business users and others marveled at the flexibility of being free from the constraints of relational data structures. Earlier (and small-scale) data lakes seemed like the perfect solution for organizations seeking agility. What companies learned though is sustainable competitive advantage requires some level of structure and their data lakes were quickly devolving into chaos. During 2019, many companies are re-launching their data lakes, adding data governance and support for relational data structures as a means of supporting agility, yet enabling enterprise-scale analytics and the sustainability of data solutions.

The Data Catalog and Metadata to Drive Consumption

One of the biggest pieces of feedback from data users (both data professionals and business consumers) is that they know the data they seek exists, but as enterprise data increases, it is becoming increasingly difficult to find that data. This is driving an increased investment in data catalogs and metadata that not only provides traditional content tagging, but also includes, for example, data quality, age and trustworthiness scores. The catalog and metadata being collected is being used to drive search capabilities as well as new AI-enabled capabilities for data correlation and advanced analysis.

IoT as the Next Wave of Big Data

Big data is no longer an emerging challenge – it is an operational reality in most companies. The new development in the big-data space is the origination of the data. More companies are deploying IoT devices, mobile apps and embedded sensors within machinery that (instead of providing large aggregated data sets) are generating large volumes of independent data streams that must be managed and reconciled. The challenge companies are facing is managing the connections between each of these independent data sources and a scalable way of integrating the data in the data warehouse. This is one of the key use-cases for IPaaS.

Hybrid Data Management

On-premises systems are not disappearing any time soon, so companies must determine how to make their on-premises and cloud data co-exist and interoperate. Many companies are addressing this by keeping their core, report-oriented data warehouses on-premises to avoid impacts to end-users while moving data staging, special data for analytic, and other pieces to a cloud-based solution. During 2019, there will be a large focus on how to manage the integrations both between different components of the data management solution as well as the interfaces with data sources and consuming applications.

Each of these trends provides a building block that will form the foundation of companies’ next generation of data management solutions. Now that individual facets of the picture have matured, companies are shifting their focus to data and solution integration, leveraging options, such as the Actian DataConnect IPaaS solution, and advanced metadata and catalog techniques to make enterprise data more integrated and more accessible to users. To learn more, visit www.actian.com/dataconnect.

About Actian Corporation

Actian makes data easy. We deliver cloud, hybrid, and on-premises data solutions that simplify how people connect, manage, and analyze data. We transform business by enabling customers to make confident, data-driven decisions that accelerate their organization’s growth. Our data platform integrates seamlessly, performs reliably, and delivers at industry-leading speeds.