Cloud-based data integration allows organizations to connect on-premises and cloud-based applications or data sources using pre-built connectors with the management infrastructure hosted by a solution provider. The solution provider manages the software, including patching and software upgrades, which reduces the IT effort for the client organization.
Cloud-based data integration can be extended to support new collaboration models such as integration-as-a-service (IaaS) and integration-platform-as-a-service (iPaaS), which make a single multi-tenant data integration system available to multiple business partners on a subscription basis.
Why is Cloud-Based Data Integration Important?
Before the advent of cloud-based data integration, businesses’ IT functions would have to acquire hardware in their data centers, procure licensed software, and install and maintain the software. As cloud-based applications became mainstream and businesses moved their existing applications and built new applications in the cloud, organizations have naturally gravitated to cloud-native infrastructure solutions. Cloud-based solutions lower management costs, increase service availability and support subscription-based pricing, eliminating the need to acquire servers, procure licenses, and maintain the software in on-premises data centers.
Key Features of a Cloud-Based Integration Solution
The capabilities of a cloud-based integration solution vary. Below are some essential features.
Data Connectors
Data integration solutions have a primary purpose of connecting disparate data sources and applications. To do this, such solutions include pre-built connectors to different data formats, including files, data streams, and applications.
Data Pipeline Orchestration
Visual workflow design tools help construct data flows from data sources to the target analytic system. A library of reusable components makes it easy to build data pipelines without coding custom scripts.
Flexible Integration Architecture
The integration platform needs to support traditional point-to-point connections in addition to bus-based and hub and spoke-based connection models.
Scheduling
Managing the movement of data across hundreds of point-to-point integrations is not scalable and is expensive to manage. Integration solutions should provide a centralized view of all data pipelines to allow IT to schedule and pause data movements.
Streamed Data
Data that needs to be shared in real-time, such as stock price changes, need to be shared as changes occur. Stream management services such as Kafka allow client applications to subscribe to data streams. Advanced integrations need to support such pull-based and pushed-based data streams.
Central Management
Managing a web of interconnected applications, data sources, and data warehouses is complex. Integration solutions can ease this process by monitoring integrations, logging exceptions, and alerting IT about failures.
Role-Based Security
The ability to delegate the creation and management of integrations based on business function and role makes cloud-based integration management more accessible for IT and the line of business that uses the integration. In the case of different business partners sharing an integration, it is essential to use strong authentication rules to govern access control. Data subject to regulatory compliance such as HIPAA or GDPR require additional security safeguards to protect privacy.
Flexible Deployment
Large enterprises can take a decade or more to complete their transition to the cloud, so they benefit from integration solutions that can be operated on-prem and across multiple public clouds.
Data Integration Use Cases
Below are real-world use cases of integration solutions provided by Actian DataConnect:
- Hannover Re uses DataConnect to collect data from regional offices around the globe and provide a consolidated view. The insights gained from the consolidated view allow the sales team to build credibility and trust from customers by sharing regional and industry-specific benchmark data.
- FICO uses DataConnect to integrate data into its internal applications.
- ADP uses DataConnect to connect to and transform data feeds from its partners and customers.
- Xactly uses DataConnect to enable its customers to pull critical data into the Xactly Incent solution for analysis.
- GE Oil & Gas uses the Actian Business Xchange, a cloud-based integration service, to streamline internal business processes and rapidly onboard trading partners to receive payments faster.
Benefits of Cloud-Based Data Integration
The benefits of cloud-based data integration include:
- Provides pre-built connectors to hundreds of file formats, application APIs, and streamed data managers.
- Simplifies management of data pipelines through centralized monitoring and administration.
- Supports subscription-based pricing that uses operational budgets.
- Eliminates the need to procure, install, and maintain integration software.
- Replaces ad hoc, point-to-point integrations with more advanced bud- and hub-based alternatives.
- Enables secure cross-company data integrations, including industry-specific exchanges.
- Provides a foundation for component-based application architectures.
Actian and Cloud-Based Data Integration
The Actian Data Platform has built-in cloud-based data integration using Actian DataConnect technology—more than 200 pre-built connectors to hundreds of data sources, including cloud-based applications such as Salesforce and NetSuite. A universal adapter makes it easy to create custom interfaces for legacy application access. Existing integration jobs can be re-used, orchestrated, and managed along with new ones.
Data can be extracted from applications and passed to the built-in data warehousing capabilities powered by the Actian Vector columnar database that delivers analytics in milliseconds with minimal administration effort. Actian Vector is self-indexing and provides superior scalability through unmatched vertical and horizontal scalability using MPP and parallel query capabilities.
