Software Defined Integration
The primary concept of software-defined networking (SDN) is “the decoupling of the system that makes decisions about where traffic is sent (the control plane) from the underlying systems that forward traffic to the selected destination (the data plane).” The Open Networking Foundation defines the SDN architecture as:
- Directly programmable
- Centrally managed
- Programmatically configured
- Open standards-based and vendor-neutral
As we outlined in this technical whitepaper, the SnapLogic Integration Cloud is architected on SDN concepts and built with a specific set of values in mind. The “control plane” controls where and how data is processed based on user configuration and preferences. The “data plane” (aka the Snaplex) does the actual processing of data as per instructions received from the control plane. No data is stored in the SnapLogic Integration Cloud, instead data is streamed between systems via the Snaplex. In this post we’ll provide an overview of the SnapLogic data plane. You can learn more about the control plane here and Snaps here. We’ll dive into security in the next post.
Snaplex: The Data Plane
A Snaplex is the data processing component of the SnapLogic Integration Cloud. It is the “data plane.” Customers can deploy one or many Snaplexes as required to run pipelines and process data. A Snaplex consists of one or more Nodes and come in two flavors ? on-premises (aka ?Groundplex?) and in the cloud (aka ?Cloudplex?).
- Cloudplex: All Cloudplexes run inside the SnapLogic Integration Cloud. Customers use the Manager and the Monitoring Dashboard to administer their Cloudplex. The SnapLogic DevOps team administers the infrastructure key performance indicators (KPIs) such as uptime, etc. Customers needing to run integrations that orchestrate across cloud applications (e.g. Salesforce, ServiceNow, Workday) with no on-premises connections will not require any software to run behind their firewall.
- Groundplex: Customers needing on-premises connectivity (e.g. SAP, Oracle, Microsoft Dynamics AX, etc.) will need a Groundplex, which runs behind the firewall. Although they run on private or virtual private data centers, Groundplexes are managed remotely by the SnapLogic Integration Cloud control plane (e.g. heartbeat monitoring, software upgrades, etc.).
The Snaplex can elastically expand and contract based on data traffic flowing through it. The unit of scalability inside Snaplex is a Java virtual machine (JVM), referred to as a ?Node?. The control plane has built-in ?smarts? to automatically scale the Snaplex out and in, in order to handle variable traffic loads. For instance, each Snaplex is initialized with a configurable minimum number of Nodes (say one, for example). Once the utilization of this one node
reaches a certain configurable threshold (say a certain number of pipelines running or a certain percentage of CPU or memory utilization per node) due to a spike in traffic, a new Node is automatically spun up to handle the additional workload. Once this excess data traffic has been processed and the second Node becomes idle, it gets ?torn down? to scale back the Snaplex to its original size.