Gaurav Dhillon on Nathan Latka’s “The Top” Podcast

Popular podcast host Nathan Latka has a built a large following getting top CEOs, founders, and entrepreneurs to share strategies and tactics that set them up for business success. A data industry veteran and self-described “company-builder,” SnapLogic founder and CEO Gaurav Dhillon was recently invited by Nathan to appear as a featured guest on “The Top.”

Nathan is known for his rapid-fire, straight-to-the-point questioning, and Gaurav was more than up to the challenge. In this episode, the two looked back at Gaurav’s founding of Informatica in the ’90s; how he took that company public and helped it grow to become a billion-plus dollar business; why he stepped away from Informatica and decided to start SnapLogic; how integration fuels digital business and why customers are demanding modern solutions like SnapLogic’s that are easy to use and built for the cloud; and how he’s building a fast-growing, innovative business that also has it’s feet on the ground.

The two also kept it fun, with Gaurav fielding Nathan’s “Famous Five” show-closing questions, including favorite book, most admired CEO, advice to your 20-year-old self, and more.

You can listen to the full podcast above or via the following links:

Case study: Connecting the dots at Box

“Data needs to be delivered to a user in the right place at the right time in the right volume.”

Spoken by veteran SnapLogic user Alan Leung, Senior Enterprise Systems Program Manager at Box, Alan provides insight in this case study for why a cloud-first ecosystem with self-service integration is the right solution for many enterprise companies. Just as Box is on a mission to improve and innovate cloud-based file storage, internally they have moved toward a cloud-centric infrastructure that benefits from a cloud-based integration platform.

connecting-the-dots-at-box

Read the full case study here or take a look at some highlights below:

  • Overall problem: Box needed to more efficiently integrate cloud-based applications, including Salesforce, Zuora, NetSuite, and Tableau, all of which they relied on for daily operations.
  • Challenges: The primary challenge was around APIs – each application’s endpoints for integration purposes behaved differently, limiting abilities to build useful connections quickly and resulting in a series of disjointed silos. Manual upload and download processes caused a strain on resources and a waste of time and effort.
  • Goal: To satisfy the need to aggregate the business data piling up in various applications into a cloud-based warehouse to enable self-service, predictive analytics.
  • Solution needed: A cloud-based integration platform that vastly reduced or eliminated the time-consuming manual processes the users faced.
  • Solution found: With the SnapLogic Elastic Integration Cloud, Alan and his team benefitted from:
    • A platform that did not require sophisticated technical skills
    • The agility to enable quick and efficient integration projects
    • The ability to handle both structured and unstructured data at speed
    • An enhanced ability to quickly analyze and make sense of so much data, allowing the company to “rapidly pivot [our] operations to seize opportunity across every aspects of the business.”

For a quick snapshot, Box currently has 23 applications connected through the platform, resulting in 170 data pipelines processing over 15 billion transactions daily. They also have eliminated the need to build a single interface internally; and an ongoing benefit of Box’s partnership with SnapLogic is that more Snaps are always being created and can be implemented for future integration needs.

Learn more about all of our customers here, and stay tuned for more customer stories.

Gartner Names SnapLogic a Leader in the 2017 Enterprise iPaaS Magic Quadrant

For the second year in a row, SnapLogic has been named a Leader in Gartner’s Magic Quadrant for Enterprise Integration Platform as a Service (iPaaS).

Gartner evaluated iPaaS vendors on “completeness of vision” and “ability to execute.” Those named to the Leaders quadrant, as Gartner noted in the report, “have a solid reputation, with notable market presence and a proven track record in enabling … their platforms are well-proven and functionally rich, with regular releases to rapidly address this fast-evolving market.”

In a press release issued today, SnapLogic CTO James Markarian said of the recognition: “Since our inception, we have been laser-focused on delivering a modern enterprise integration platform that is specifically designed to manage the data and application integration demands of today’s hybrid enterprise technology environments. Our Enterprise Integration Cloud eliminates the complexity of legacy integrations, providing a platform that supports fast and easy self-service integration.”

The Enterprise iPaaS Magic Quadrant is embedded below. We’d encourage you to download the complete report as it provides a comprehensive review of all the vendors and the growing market.

Gartner 2017 iPaaS MQ

Thanks to all of SnapLogic’s customers, partners, and employees for the ongoing support and for making SnapLogic’s Enterprise Integration Cloud a leading self-service integration platform connecting applications, data, and things.

Podcast: James Markarian and David Linthicum on New Approaches to Cloud Integration

SnapLogic CTO James Markarian recently joined cloud expert David Linthicum as a guest on the Doppler Cloud Podcast. The two discussed the mass movement to the cloud and how this is changing how companies approach both application and data integration.

In this 20-minute podcast, “Data Integration from Different Perspectives,” the pair discuss how to navigate the new realities of hybrid app integration, data and analytics moving to the cloud, user demand for self-service technologies, the emerging impact of AI and ML, and more.

You can listen to the full podcast here, and below:

 

VIDEO: SnapLogic Discusses Big Data on #theCUBE from Strata+Hadoop World San Jose

It’s Big Data Week here in Silicon Valley with data experts from around the globe convening at Strata+Hadoop World San Jose for a packed week of keynotes, education, networking and more - and SnapLogic was front-and-center for all the action.

SnapLogic stopped by theCUBE, the popular video-interview show that live-streams from top tech events, and joined hosts Jeff Frick and George Gilbert for a spirited and wide-ranging discussion of all things Big Data.

First up was SnapLogic CEO Gaurav Dhillon, who discussed SnapLogic’s record-growth year in 2016, the acceleration of Big Data moving to the cloud, SnapLogic’s strong momentum working with AWS Redshift and Microsoft Azure platforms, the emerging applications and benefits of ML and AI, customers increasingly ditching legacy technology in favor of modern, cloud-first, self-service solutions, and more. You can watch Gaurav’s full video below, and here:

Next up was SnapLogic Chief Enterprise Architect Ravi Dharnikota, together with our customer, Katharine Matsumoto, Data Scientist at eero. A fast-growing Silicon Valley startup, eero makes a smart wireless networking system that intelligently routes data traffic on your wireless network in a way that reduces buffering and gets rid of dead zones in your home. Katharine leads a small data and analytics team and discussed how, with SnapLogic’s self-service cloud integration platform, she’s able to easily connect a myriad of ever-growing apps and systems and make important data accessible to as many as 15 different line-of-business teams, thereby empowering business users and enabling faster business outcomes. The pair also discussed ML and IoT integration which is helping eero consistently deliver an increasingly smart and powerful product to customers. You can watch Ravi and Katharine’s full video below, and here:

 

Cloud fluency: Does your data integration solution speak the truth?

There’s been a lot of cloud-washing in the enterprise data integration space — vendors are heavily promoting their cloud solutions, yet for many, only a skinny part of their monolithic apps has been “cloudified.”

In an era of “alternative facts,” it’s important to make technology decisions based on truths. Here is an important one: A big data integration solution built on genuine, made-for-the-cloud platform as a service (PaaS) technology offers important benefits including:

  1. Self-service integration by “citizen integrators,” without reliance on IT
  2. For IT organizations, the ability to easily connect multiple data sets, to achieve a bespoke enterprise tech environment

These are in addition to the traditional benefits of cloud solutions: no on-premise installation; continuous, no-fuss upgrades; and the latest software innovation, delivered automatically.

Why “built for the cloud” matters

You can’t get these benefits with “cloudified” software that was originally invented in 1992. Of course, I’m referring to Informatica; while the company promotes its cloud capabilities, the software largely retains a monolithic architecture that resides on-premises, and does most of its work there, too.

In contrast, SnapLogic is purpose-built for the cloud, meaning there are no legacy components that prevent the data and application integration service from running at cloud speed. Data streams between applications, databases, files, social and big data sources via the Snaplex, a self-upgrading, elastic execution grid.

In more everyday terms, SnapLogic has 100% cloud fluency. Our technology was made for the cloud, born in the cloud, and it lives in the cloud.

The consumerization of data integration

Further to point 1 above, “citizen integrators,” industry futurists like R. “Ray” Wang have been talking about the consumerization of IT for more than half a decade. And that is exactly what SnapLogic has mastered. Our great breakthrough, our big innovation, is that we have consumerized the dungeon-like, dark problem of data integration.

Integration used to be a big, boring problem relegated to the back office. We’ve brought it from the dungeon to the front office and into the light. It is amazing to see how people use our product. They go from one user to hundreds of users as they get access to data in a secure, organized and appropriately access-controlled manner. But you don’t have a cast of thousands of IT people enabling all this; users merely help themselves. This is the right model for the modern enterprise.

“An ERP of one”

As for the second major benefit of a true cloud solution — a bespoke enterprise tech environment, at a fraction of the time and cost of traditional means — here’s a customer quote from a young CEO of a hot company that’s a household name.

“Look, we’ve got an ‘ERP of one’ by using SnapLogic — a totally customized enterprise information environment. We can buy the best-of-the-best SaaS offerings, and then with SnapLogic, integrate them into a bespoke ERP system that would cost a bajillion dollars to build ourselves. We can custom mix and match the capabilities that uniquely fit us. We got the bespoke suit at off-the-rack prices by using SnapLogic to customize our enterprise environment.”

To my mind, that’s the big payoff, and an excellent way to think about SnapLogic’s value. We are able to give our customer an “ERP of one” faster and cheaper than they could have ever imagined. This is where the world is going, because of the vanishingly low prices of compute power and storage, and cloud computing.

Today you literally can, without a huge outlay, build your own enterprise technology world. But you need the glue to realize the vision, to bring it all together. That glue is SnapLogic.

Find out more about how and why SnapLogic puts best-of-breed enterprise integration within every organization’s grasp. Register for this upcoming webinar featuring a conversation with myself, industry analyst and data integration expert David Linthicum, and Gaurav Dhillon, SnapLogic’s CEO and also an Informatica alumnus: “We left Informatica. Now you can, too.”

SNP_Thumb_Informatica

Gaurav Dhillon is CEO at SnapLogic. Follow him on Twitter @gdhillon.

Deep Dive into SnapLogic Winter 2017 Snaps Release

By Pavan Venkatesh

Data streams with Confluent and migration to Hadoop: In my previous blog post, I explained how future data movement trends will look. In this post, I’ll dig into some of the exciting things we announced as part of the Winter 2017 (4.8) Snaps release. This will also address future data movement trends for customers who want to move data to the cloud from different systems or migrate to Hadoop.

Major highlights in 2017 Winter release (4.8) include:

  • Support of Confluent Kafka – A distributed messaging system for streaming data
  • Teradata to Hadoop – A quick and easy way to migrate data
  • Enhancements to the Teradata Snap Pack: On the TPT front, customers can quickly load/update/delete data in Teradata
  • The RedShift Multi-Execute Snap – Allows multiple statements to be sequentially executed, so customers can maintain business logic
  • Enhancements to the MongoDB Snap pack (Delete and Update) and the DynamoDB Snap pack (Delete and Delete-item)
  • Workday Read output enhancements – Now it’s easier for the downstream systems to consume
  • Netsuite Snap Pack improvements -Users can now submit asynchronous operations
  • Security feature enhancements – Including SSL for MongoDB Snap Pack and invalidating database connection pools when account properties are modified
  • Major performance improvement while writing to an S3 bucket using S3 File Writer – Users can now configure a buffer size in the Snap so larger blocks are sent to S3 quickly

Confluent Kafka Snap Pack

Kafka is a distributed messaging system based on publish/subscribe model with high throughput and scalability. It is mainly used for ingestion from multiple sources and then sent to multiple downstream systems. Use cases include website activity tracking, fraud analytics, log aggregation, sales analytics, and others. Confluent is the company that provides the enterprise capability and offering for open source Kafka.

Here at SnapLogic we have built Kafka Producer and Consumer Snaps as part of the Confluent Snap Pack. A deep dive into Kafka architecture and its working will be a good segue before going into the Snap Pack or pipeline details.

kafka-cluster

Kafka consists of single or multiple Producers that can produce messages from a single or multiple upstream systems, and single or multiple Consumers that consume messages as part of downstream systems. A Kafka cluster constitutes one or more servers called Brokers. Messages (key and value or just the value) will be fed into higher level abstraction called Topics. Each Topic can have multiple messages from different Producers. User can also define different Topics for new category of messages. These Producers write messages to Topics and Consumers consume from one or more Topics. Also Topics are partitioned, replicated, and persisted across Brokers. Messages in the Topics are ordered within a partition and each of these will have a sequential ID number called offset. Zookeeper usually maintains these offsets but Confluent calls it coordination kernel.

Kafka also allows configuring a Consumer group where multiple Consumers are part of it, when consuming from a Topic.

With over 400 Snaps supporting various on-prem (relational databases, files, nosql databases, and others) and cloud products (Netsuite, SalesForce, Workday, RedShift, Anaplan, and others), the Snaplogic Elastic Integration Cloud in combination with the Confluent Kafka Snap Pack will be a powerful combination for moving data to different systems in a fast and streaming manner. Customers can realize benefits and generate business outcomes in a quick manner.

With respect to the Confluent Kafka Snap Pack, we support Confluent Version 3.0.1 (Kafka v0.9). These Snaps abstract the complexities and users only have to provide configuration details to build a pipeline which moves data easily. One thing to note is that when multiple Consumer Snaps are used in a pipeline and have been configured with the same consumer group, then each Consumer Snap will be assigned a different subset of partitions in the Topic.

kafka-producer

kafka-consumer

pipeline1

In the above example, I built a pipeline where sales leads (messages) stored in local files and MySQL are sent to a Topic in Confluent Kafka via Confluent Kafka Producer Snaps. The downstream system Redshift will consume these messages from that Topic via the Confluent Kafka Consumer Snap and bulk load it to RedShift for historical or auditing needs. These messages are also sent to Tableau as another Consumer to run analytics on how many leads were generated this year, so customer can compare this against last year.

Easy migrations from Teradata to Hadoop

There has been a major shift where customers are moving from expensive Teradata solutions to Hadoop or other data warehouse. Until now, there has not been an easy solution in transferring large amounts of data from Teradata to big data Hadoop. With this release we have developed a Teradata Export to HDFS Snap with two goals in mind: 1) ease of use and 2) high performance. This Snap uses the Teradata Connector for Hadoop (TDCH v1.5.1). Customers just have to download this connector from the Teradata website in addition to the regular jdbc jars. No installation required on either Teradata or Hadoop nodes.

TDCH utilizes MapReduce (MR) as its execution engine where the queries gets submitted to this framework, and the distributed processes launched by the MapReduce framework make JDBC connections to the Teradata database. The data fetched will be directly loaded into the defined HDFS location. The degree of parallelism for these TDCH jobs is defined by the number of mappers (a Snap configuration) used by the MapReduce job. The number of mappers also defines the number of files created in HDFS location.

The Snap account details with a sample query to extract data from Teradata and load it to HDFS is shown below.

edit-account

terradata-export

 

The pipeline to this effect is as follows:

pipeline2

As you can see above, you use just one Snap to export data from Teradata and load it into HDFS. Customers can later use HDFS Reader Snap to read files that are exported.

Winter 2017 release has equipped customers with lots of benefits, from data streams, easy migrations, to enhancing security functionality, and performance benefits. More information on the SnapLogic Winter 2017 (4.8) release can be found in the release notes.

Pavan Venkatesh is Senior Product Manager at SnapLogic. Follow him on Twitter @pavankv.