Guaranteed delivery with SnapLogic Resumable Pipelines

3 min read

The SnapLogic team takes pride in continuously enhancing the SnapLogic Intelligent Integration Platform (IIP) to encapsulate multiple execution paradigms for the different needs and use cases of the enterprise.

One of the most popular use cases within the IIP is a standard execution that enables both scheduled batch type jobs along with event-driven ones. Both of these jobs provide a low execution overhead which can be deployed in the cloud or on-premises.

The SnapLogic Pipeline journey

Working from the success of the IIP’s original scheduled batch and event-driven jobs, we partnered with customers to answer their needs around supporting low-latency transactional-type executions. Enter the SnapLogic Ultra Pipeline execution paradigm. SnapLogic Ultra Pipelines essentially pre-caches the execution context with an added level of at-least-once-guaranteed delivery so business-critical messages are not lost. 

Eventually, the demand increased for exactly-once delivery semantics in a given pipeline execution, and SnapLogic Resumable Pipelines were born. Unlike SnapLogic Ultra Pipelines, Resumable Pipelines eliminate the possibilities of data duplication with exactly-once delivery (i.e., duplication of invoices). Resumable Pipelines do not replace Ultra Pipelines but serve a different set of business use cases that you will leverage when completing integration projects. SnapLogic Ultra Pipelines will continue to be the leader for low-latency executions.

Suspended pipelines can be corrected to resume execution without having to restart the process from the beginning
Suspended pipelines can be corrected to resume execution without having to restart the process from the beginning

With SnapLogic Resumable Pipelines, exactly-once semantics for scheduled and triggered executions are available on any Snaplex you choose. Pipelines will save state, and on failure, they will be “suspended.” From there, suspended pipelines can be “resumed” where documents would resume from the point of failure. Additionally, some of the potential challenges of needing a feed-master (for event-driven Ultra Pipelines) or blocking aggregating type functions are no longer the case with Resumable Pipelines.

Snaps in suspended pipelines are color-coded orange to indicate where the documents stopped processing.
Snaps in suspended pipelines are color-coded orange to indicate where the documents stopped processing.

For more information about Resumable Pipelines, please check the Resumable Pipeline video and SnapLogic documentation. We look forward to making additional iterations and enhancements to SnapLogic Resumable Pipelines.

Former Sr Product Manager at SnapLogic

We're hiring!

Discover your next great career opportunity.