In this series of posts I’m reviewing 6 essential integration platform as a service (iPaaS) ingredients, as outlined in this post: What to Look for in a Modern Integration Platform. I covered why it’s important that the iPaaS be a fully-functional cloud-based service (based on a software-defined architecture) here. In this post I’ll cover the requirement for a single platform for data, apps and API integration.
We’ve written a lot about the so-called Integrator’s Dilemma faced by enterprise IT organization whose lightweight SaaS apps are being weighed down by their heavy-weight data and application integration tools that were built well before era of big data, social, mobile and cloud computing (SMAC). One CIO I spoke with recently referred to his legacy technologies for connecting SaaS and on-premises applications with each other (and all of his companyies’ disparate data sources) as “boat-anchor integration.” The good news is that innovation is back in the integration market and as sure as death and taxes, disparate silos of integration will converge in the cloud under the more agile iPaaS umbrella. At SnapLogic, we’re talking to customers and prospects about the benefits of an integration platform that is software-defined and built from the ground up to handle batch and streaming requirements. Yes, these can be for very different use cases:
- Batch data integration is typically for getting data in and out of a SaaS app (initial load or scheduled replication) for analytical requirements. The industry has historically called this capability: extraction, transformation and load (ETL), which extends to data cleansing and quality as well as master data management (MDM). The relevance of traditional ETL approaches are now being questioned as Hadoop deployments grow and legacy tools in this category were built to handle rows and columns and structured data integration requirements. A few months ago we ran a popular webinar with Dave Linthicum called Beyond Batch: Is ETL Dead? At the recent Hadoop Summit, the relationship between the data warehouse, SQL and Hadoop was one of the hottest topics. Check out the tweet stream or headlines like this one: Vendors fret as Hadoop encroaches on lucrative data warehouse business.
- Real-time streaming integration is typically for more operational than analytical use cases. Common SaaS application integration uses cases are connecting front-office apps like Salesforce and Workday with back-office financial systems like SAP, Oracle EBS and NetSuite. Typically event-based, SaaS application integration continues to require features like Guaranteed Delivery, but the need to adapt quickly to frequent changes, API updates and the demand for end-user self-service has made legacy approaches inadequate. XML and SOAP-based integration tools in this category were only designed to handle small data – high-frequency, low-latency messages. We’ve written about why the heavy-weight enterprise service bus (ESB) will become less and less relevant for today’s agile iPaaS requirements in this whitepaper – Thoughts on ESBs: Why Buses Don’t Fly in the Cloud. You can also read about the technical advantages of a JSON-centric iPaaS here.
So as we continue to see the convergence of historically distinct integration categories (data, application, process), increasingly we believe a single platform will be the right approach and as such it is one of the primary requirements for a modern iPaaS solution. In my next post I’ll talk about Elastic Scale. In the meantime, you can check out a demonstration of the SnapLogic Elastic Integration Platform here.