In this series of posts, I’ve been sharing excerpts from a recent discussion between SnapLogic founder and CEO Gaurav Dhillon and John Gallant, chief content officer of IDG US Media. In the first post, Gaurav discussed the looming integration issues that people should focus on. In the second post, Gaurav reviewed the specific challenges SnapLogic addresses. In this post, Gaurav describes the core components and principles of the SnapLogic Elastic Integration Platform. Continue reading “Gaurav Dhillon Q&A: SnapLogic Elastic Integration Overview”
SnapLogic’s Vice President of Engineering Vaikom Krishnan recently outlined the 10 New Requirements for Modern Data Integration. Here is a presentation that summarizes these requirements.
As the integration market continues to mature, there is a constant demand to support and process more complex data and process flows. When applications process large data, they often run out of resources and become unresponsive, leaving users confused and unhappy. Gauging resources and alerting users with appropriate messages are some of the most important factors of ideal software. In the Winter 2016 release of the SnapLogic Elastic Integration Platform, we introduced the concept of pipeline queuing, which allows users to define thresholds for their Snaplexes, and when thresholds are reached, any further requests to it are queued until the next resources are available. Continue reading “Snaplex Thresholds and Pipeline Queuing”
Welcome to Puzzle Pieces, a periodic series exploring the “Why?” of SnapLogic’s platform. To kick things off, let’s talk Snaplexes, which have sometimes proved puzzling. (Editor’s note: future installments of Puzzle Pieces will be rigorously scrubbed for alliterative excesses).
The SnapLogic Elastic Integration Platform is divided into two main parts: the Control Plane and the Data Plane. As a customer, you come into contact with the Control Plane through the SnapLogic web interface. Behind the scenes, the Control Plane also handles talking to the Data Plane and coordinating the flow of data in pipelines.
The pipelines actually run in the Data Plane. The container that handles running a particular pipeline is called a Snaplex. A Snaplex (or Plex) is a collection of computing resources – perhaps one virtual machine, perhaps an entire server rack. These are the Snaplex types you may come across:
In this video, Jim Teal from iRobot talks about how SnapLogic allows his organization to be more agile and operate as a real-time business supporting their global manufacturing operation.
“It’s the ease of use and flexibility of the platform that really is a huge accelerator in productivity.”
SnapLogic was recently reviewed by advisory firm MWD Advisors for our efforts to reinvent integration platform technology by creating one unified platform that can address many different kinds of application and data integration use cases.
A few highlights from the report:
- An in-depth look at our multi-tenant, AWS-hosted platform which includes the SnapLogic Designer, Manager and Dashboard
- The Snaplex execution environment; namely, the Cloudplex, Groundplex and Hadooplex
- 3 competitive differentiators – deployment flexibility, a unified approach across multiple integration scenario types and both scalability and adaptability
MWD Advisors is a specialist advisory firm providing practical industry insights to business leaders and technology professionals working to drive change with the help of digital technology.
I wrote a summary of the 10 New Requirements for Modern Data Integration that was published today on the Database Trends and Applications website. They are:
- Application integration is done primarily through REST and SOAP services
- Large-volume data integration is available to a Hadoop-based data lake or to cloud-based data warehouses
- Integration has to support the continuum of data velocities starting from batch all the way to continuous streams
- Integration is event-based rather than clock-driven
- Integration is primarily document-centric
- Integration is hybrid and spans cloud-cloud and cloud-ground scenarios
- Integration itself has to be accessible through SOAP/REST APIs
- Integration is all about connectivity, connectivity, connectivity
- Integration has to be elastic
- Integration has to be delivered as a service
These new requirements have given rise to a new category of enterprise integration called integration platform as a service (iPaaS), which should be built from the ground up to address the new and legacy enterprise application and data integration needs. As I noted in our recent SnapLogic Elastic Integration Platform Winter 2016 announcement:
“Integration must run at the speed of business and it must be unified. Whether you’re moving to the cloud or re-thinking your data architecture with Spark and Hadoop, there’s never been a better time to re-think how you’re going to tackle the age-old problem of connecting your applications and data at scale.”
I encourage you to read the full DBTA article and to share your feedback with me.