I have had the privilege of working for two of the most successful technology companies in the data space – Snowflake Computing [data warehousing as a service (DWaaS)] and SnapLogic [integration platform as a service (iPaaS)]. These experiences have been really valuable in shaping me as a data professional. As a result of working for these great companies, I now laugh at the notion that implementing a data-driven initiative is difficult and expensive.
Snowflake: A powerhouse of performance and cost savings
Snowflake’s separation of computing and storage and its usage-based billing deliver a powerhouse of performance and cost savings. On one hand, it allows data engineers and end-users to harmoniously operate on their data without fighting for computing resources. One user can easily be querying for terabytes (TB) of data while another is loading the same amount against the same table. Yet, none of these users will see a drop in performance – data loading and reading across the same data is no longer an issue. Users can instantaneously increase compute power to finish their work without provisioning new hardware or spinning up and down different virtual machines (VMs). In fact, it requires just a click of a button, and Snowflake takes care of the rest. The best part is that users only pay for what they use – nothing more, nothing less.
With such a powerful tool, pushing it to its full potential requires an equally powerful data integration platform. This is especially true if the project requires multiple sources and a mixture of structured and semi-structured data types. Moreover, large enterprises typically need to integrate both on-premises and cloud resources. I’ve seen a lot of integration tools buckle at the knees when facing this menagerie of requirements. Fortunately, this is where SnapLogic shines.
SnapLogic: One platform for all your application and data integrations
The SnapLogic Intelligent Integration Platform (IIP) enables you to connect to any data type, any time, at any speed, anywhere in a single platform. It focuses on data-in-motion via a streaming-based architecture and a simple user interface that delivers data from multiple sources at once. Individual pre-built connectors, called Snaps (akin to Lego pieces for integration), enable users to sketch, test, and productize their workflow in a matter of minutes. Since the platform uses a semi-structured data type in its core, users can quickly mold and shape any type of data structure from the source to its destination, regardless of the source type, while maintaining data integrity.
Snowflake is more powerful with SnapLogic
SnapLogic’s Snowflake Snap Pack was developed with consideration and care. It complements Snowflake’s strengths and adapts to its nuances. The Bulk Load Snaps take advantage of Snowflake’s COPY command by automatically staging incoming data in either internal or external stages. The Multi Execute Snap respects Snowflake’s auto-commit nature by giving users finer control in managing transactions. The SCD Type 2 Snap extends Snowflake’s change data capture capability.
SnapLogic extends the Snowflake Snap Pack even further by enabling connections from on-premises databases like MySQL, PostgreSQL, MS SQL, Oracle DB, Teradata, and Netezza. At the same time, you can pull data from on-premises or cloud applications, REST or SOAP resources, or file systems – I’m talking all at once, concurrently, without writing a single line of code! On top of this, SnapLogic’s workflow management layer allows you to quickly deploy, monitor, and troubleshoot workflows within a single view. The native REST API management layers can turn all your integrations into RESTful endpoints for other applications to consume. It’s a genuine unified platform for all your integration needs.
Popular use cases for SnapLogic and Snowflake
Recently, I’ve been going through unique use cases involving SnapLogic and Snowflake. I created a series of videos showcasing unique possibilities for combining both tools. Feel free to check them out.
- Snowflake Teradata Migration
- Snowflake SCD 2
- Snowflake LZO Compression
- Snowflake DDL Version Control
SnapLogic is tightly integrated with Snowflake. It takes the complexity out of managing Snowflake connectors and enriches the user’s experience with the most versatile set of ETL and ELT operations available. Gain insight into your business faster by delivering data to Snowflake with SnapLogic. Snowflake users can easily access SnapLogic via the Snowflake Partner Connect portal. Visit the portal now to access all of SnapLogic’s Snowflake and other Snap Packs!
