SnapLogic iPaaSWith Gartner’s 2nd Enterprise Integration Platform as a Service (iPaaS) report published last week, Loraine Lawson at ITBusinessEdge has referred to the iPaaS market as a “Volatile Boom Town.” At SnapLogic, we’re proud to announce that we have once again been recognized as a Visionary Leader in this high-growth market. Here’s what our founder and CEO, Gaurav Dhillon, had to say about the report:

“Since launching the SnapLogic Elastic Integration Platform in late 2013 our product development team has been laser-focused on adding new functionality that solves the challenges of integrating today’s hybrid environments with on-premise, cloud and big data applications. We believe our visionary position is a reflection of growing enterprise customer adoption an continued innovation in delivering a unified platform that addresses both application and big data integration use cases simultaneously.”

Gaurav also talked about how SnapLogic’s cloud integration bet paid off in this diginomica featured interview last week.

At SnapLogic, we’ve set up to address the Integrator’s Dilemma, by pushing for the convergence of application and data integration in the modern era. It’s time to rethink legacy ESB and ETL in the enterprise. Here are some of the companies highlights in the last 12 months. Thanks to all of our customers and partners for their continued support.

Company Execution:

Platform Innovations

  • 300+ Snaps: pre-built connectors, called Snaps, to the most popular cloud and big data systems are now available, including Box, Cloudera, Concur, Eloqua, Hortonworks, NetSuite, MongoDB, Salesforce, SAP, ServiceNow and Workday
  • Snap Patterns: reusable process and data integration patterns make it easy to configure and address the most common integration use cases and data flows for connecting cloud services like Amazon Redshift from Amazon Web Services
  • SnapReduce 2.0: translates data flow pipelines into MapReduce code, enabling customers to ingest, prepare and deliver big data to power next generation analytics
  • Hadooplex: SnapLogic’s hybrid execution grid can run in the cloud, on-premise or as a native YARN application in a Hadoop cluster
  • Ultra Pipelines: dramatically speeds the process by which time-sensitive data is delivered to cloud and big data applications

Partnership Momentum

  • Cloudera and Hortonworks certifications: the Hadooplex introduced to run as a Hadoop 2.0 YARN application
  • SAP Netweaver certification: enables customers to easily share data between their SAP software and other enterprise and cloud applications. SAP Hana Snaps also introduced
  • Integration with Okta: provides seamless, secure access to the SnapLogic platform and a single point of control for user authentication and authorization through the Okta Application Networks
  • AWS Redshift Free Trial and Snap Patterns: quickly build, deploy and efficiently manage multiple high-volume, data-intensive integration tasks delivering data to and from a cloud data warehouse
  • Salesforce Analytics Cloud Snaps and Patterns: powering the Wave with easy data loading and transformation

As you can see it was a busy year for the SnapLogic team…and we’re just getting started! This week we’ll be at Gartner BI in Las Vegas and on April 9th, we’ll be at the AWS Summit in San Francisco. You can check out all of our upcoming events here and be sure to Contact Us if you’d like more information on the SnapLogic Elastic Integration Platform.

The Enterprise Service Bus (ESB) is a model for inter-application communication that was developed in the era of the Service Oriented Architecture (SOA). Today when people mention the term ESB, the first things that typically come to mind are long, expensive development cycles, inflexibility, and complexity. While designed with good intentions, most legacy ESB implementations grew into overloaded business logic monsters, causing performance issues, especially when attempting to deal with large volumes of data. What people ultimately want from an ESB is a high-speed, agile, and scalable infrastructure that is ubiquitous, able to work with modern data and services both on-premises and in the cloud.

At the heart of legacy ESB technology are the Simple Object Access Protocol (SOAP) and eXtensible Markup Language (XML), which are both becoming outdated, designed at a time when the Service Oriented Architecture (SOA) was mainstream. In the new enterprise IT world of the SMACT (social, mobile, analytics, cloud and things), especially when it comes to big data, message-based ESB technology begins looking more like a horse and buggy than a modern approach to application and data integration. The application and data worlds have evolved and the requirements have converged. In that evolution the ESB is no longer a viable option.

no_esbSnapLogic’s answer to low-latency application and data integration lies in a high-speed, guaranteed delivery system that works across disparate sources, whether on-premise or in the cloud. Designed for high-speed data streaming, transformation and delivery, SnapLogic’s Ultra Pipelines are designed to meet the demands of today’s modern systems, supporting virtually any type of interface or data representation (JSON, XML, SOAP, REST, etc.) with low-latency, real-time processing, and guaranteed data delivery.

SnapLogic has removed the heavy lifting typically found in ESB systems and implementations, moving the user from a long and costly development paradigm into a quick configuration paradigm. Whether your requirement is event-based, real-time, streaming or scheduled batch-oriented data integration, SnapLogic’s modern, JSON-centric, RESTful platform has a clear advantage over traditional ESB solutions that were built for last generation’s structured application and data management challenges.

Ultra-PipelinesGraphic_NoBG (1)But what about microservices? As the enterprise continues to migrate more toward increasingly granular services, all the more reason to adopt an agile, modern integration platform that is equipped to deal with evolutionary changes. SnapLogic’s concept of the ESB rests on a system that was designed from the ground up to deliver the ultimate in flexibility, agility, and elasticity.  As EMA Research recently noted:

“Even as pre-built Snaps eliminate work cycles and manual processes related to data integration, SnapLogic Ultra Pipelines accelerates data delivery. This parallel approach to addressing the twin challenges of big data integration and real-time processing reinforces SnapLogic’s position as a leader in the data integration and iPaaS space.”

Randy Hamilton is a Silicon Valley entrepreneur and technologist who writes periodically about industry related topics including the cloud, big data and iOT.  Randy has held positions as Instructor (Open Distributed Systems) at UC Santa Cruz and has enjoyed positions at Basho (Riak NoSQL database), Sun Microsystems, and Outlook Ventures, as well as being one of the founding members and VP Engineering at Match.com.

SnapLogic_is_HiringBetween our January announcement of triple bookings in 2014 and hiring a new head of sales in February, SnapLogic has had an exciting past year and we are continuing to grow. Check out our Jobs page for all open positions and see which teams are hiring below:

  • Sales
  • Marketing
  • Customer Success
  • Technical Support
  • Professional Services
  • Solutions Engineers
  • Engineering (DevOps, Software, UX, QA)

All open positions are available here.

Screen Shot 2015-03-24 at 2.57.50 PMSo why work at SnapLogic, other than for the opportunity to learn and grow at a company responsible for cutting edge technology? As each team within the company grows, SnapLogic continues to offer a great variety of employee benefits and perks. Along with weekly “Snappy Hours,” catered meals, team off-sites, company events and competitive insurance plans (plus others below), a full list of company offerings and other HR benefits can be found on our Life at SnapLogic page.

We also offer learning and development opportunities to employees, as well as participation in the “SnapStar” program, which facilitates monthly peer-to-peer recognition for our core values of Customer Success, Teamwork, Innovation and Positive Attitude. To learn more about the SnapStar program, check out this case study.

Health, Wellness & Perks:

  • Health insurance (medical, dental, vision)
  • Commuter benefits
  • Pre-IPO stock options
  • 401(k) plan
  • 15 days of PTO and 11 paid holidays
  • Allowance for monthly gym membership
  • Catered lunches on Tuesdays and Thursdays, breakfast on Wednesdays
  • “Snappy Hours” every Friday
  • Endless snacks and beverages in our fully-stocked kitchen
  • Company events and volunteer opportunities

For even more information on the latest that’s happening at SnapLogic, take a look at our latest press releases and recent news coverage to learn about SnapLogic as the leader in cloud and big data integration.

snaplogic_ultra_pipelineSnapLogic recently introduced Ultra Pipelines, which is a turbo boost to our JSON-centric data pipeline technology, enabling reliable, real-time completion of time-sensitive enterprise tasks and processes. In this Integration Developer News review, SnapLogic product manager Tim Lui describes Ultra Pipelines this way:

“Unlike some iPaaS or on-prem integration solutions that require a trigger or an event (such as a write or update) to activate an integration, our real-time integration technology puts in place a long-running connection that is always listening and waiting for instructions. That way, Ultra Pipelines are instantly ready to execute when there is new data.”

He goes on to note, “something we find customers really like is they gain this added speed — without having to give up scalability. So, customers gain huge performance benefits and still get the elastic architecture we’ve always offered.”

In a comprehensive article about the advantages of lightweight “microservices” compared to “the heavyweight, middleware-centric enterprise service buses (ESBs) that drove many of the SOA implementations in the 2000s,” SOA author and industry expert Jason Bloomberg talks about Avoiding Dumb Pipes, observing,“SnapLogic customers can leverage their Ultra Pipelines to build microservices that serve as “web scale pipes.” (His article has started an interesting dialog on LinkedIn.)

Enterprise Management Associates also published a write up on Ultra Pipelines (SnapLogic Accelerates iPaaS Data Delivery with Ultra Pipelines), outlining how the Elastic Integration Platform is well-suited for real-time and batch-oriented application and data integration use cases. EMA concludes:

“Even as pre-built Snaps eliminate work cycles and manual processes related to data integration, SnapLogic Ultra Pipelines accelerates data delivery. This parallel approach to addressing the twin challenges of big data integration and real-time processing reinforces SnapLogic’s position as a leader in the data integration and iPaaS space.”

I’ve embedded the EMA review below. You can learn more about SnapLogic Ultra Pipelines here and for additional thoughts on why the ESB is wrong approach for managing modern cloud and big data integration requirements, be sure to read our whitepaper: Why Buses Don’t Fly in the Cloud as well as Dave Linthicum’s paper: The Death of Traditional Integration:How the Changing Nature of IT Mandates New Approaches and Technologies.

We all realize by now that our corporate data is exploding in volume, velocity and variety. We’ve also witnessed the expansion of new data sources and the resulting business insights, which has created the demand for data sources with longer histories. Legacy data transformation systems and development groups are being pushed to their limits to effectively deal with these challenges.

The concept of extract, load, and transform (ELT) was introduced to alleviate the problems associated with loading high volumes of raw data directly into a data warehouse staging area where it could be processed and transformed after loading. But now we’re realizing that traditional ETL tools simply aren’t cutting it any longer. Based on point-to-point, row/column architectures, traditional ETL tools are now at a crossroads as they struggle with huge volumes of real-time, unstructured and hierarchical data. Let’s face it folks, traditional ETL solutions are just too expensive, they don’t scale, they’re too rigid, and they require too much maintenance.

snaplogic_ipaasSnapLogic takes a truly innovative approach to data integration that is focused on data streams between applications and disparate data sources with the flexibility to connect to both cloud and on-premises systems. Delivered as a multi-tenant cloud service with a hybrid data processing engine that scales out, the SnapLogic Elastic Integration Platform takes advantage of powerful parallel processing and management capabilities with a simple-to-use, drag-and-drop Designer.  With 300+ prebuilt Snaps, you can build pipelines for data flows from any number of sources to any number of destinations. And because the SnapLogic iPaaS is 100% REST-based, pipelines are abstracted and addressable, usable, consumable, trigger-able and schedule-able as REST calls. This means they’re able to do the job of many traditional static integrations with considerable advantage. Whether one-to-one, one-to-many, many-to-one, or many-to-many orchestration scenarios, Cloud ETL challenges disappear because enterprise-grade scalability, simplicity and reliability promote fast implementations while dramatically lowering costs.

But what about real-time analytics and insights? As the demand for real-time analytics and insights increases, it’s becoming increasingly vital for data scientists and business analysts to have more control over the entire data lifecycle. Adds, moves and changes in the integration cycle need to be easily orchestrated, while preserving the time required for analytics and insights. Accomplishing this requires a modern, agile and simplified approach to ETL/ELT.

snaplogic_elastic_integrationBig Data Integration as a Service
Hadoop is a widely accepted distributed processing platform that has tools that can extract, transform and load data from high volumes of disparate data sources, but data ingest on Hadoop is a major challenge, typically requiring complex, time consuming and costly programming and scripting frameworks. Automation, metadata management and on-going maintenance in Hadoop is equally challenging.

SnapLogic has taken a novel approach to solving the Hadoop ingest challenge by moving the extraction process beyond structured data sets, allowing queries across disparate data types and structures, and then streaming all data as JSON documents. Rather than traditional point-to-point data loading, SnapLogic’s horizontally scalable elastic pipeline provides powerful multi-point, multimodal integration while hiding the underlying complexity of data integration from the user.

A big plus in the Hadoop equation is Snaplogic’s version of MapReduce called SnapReduce. As with SnapLogic’s theme of simplifying everything, SnapReduce allows the data scientist/business analyst to run MapReduce operations without special training and without the complexity of YARN-based MapReduce scripts and algorithms. With SnapLogic, the data scientist/business analyst is truly (and fully) empowered. You can think of it as “Hadoop for Humans.”

Randy Hamilton is a Silicon Valley entrepreneur and technologist who writes periodically about industry related topics including the cloud, big data and iOT.  Randy has held positions as Instructor (Open Distributed Systems) at UC Santa Cruz and has enjoyed positions at Basho (Riak NoSQL database), Sun Microsystems, and Outlook Ventures, as well as being one of the founding members and VP Engineering at Match.com.

Next Steps:

Michael Pearl is an Advisory principal and Global Cloud Computing leader with PwC. We talk with him today about a new collaboration between the companies that helps organizations transform their IT and integration strategies for the digital age.

SnapLogic: Tell us about the kind of work that PwC is doing right now in the area of integration?
PWC LogoToday’s organizations are no longer isolated enterprises—rather they are diverse and distributed ecosystems of employees, customers, partners and suppliers with different processes and systems. We are working with our clients to address these complex integration challenges through an Integration Fabric—which can deliver seamless, secure and flexible interoperability across a diverse and distributed ecosystem.

Our Integration Fabric framework is a combination of tools, accelerators, and business transformation to help organizations create, implement and manage their Integration Fabric. Our Integration Fabric framework is a cornerstone of our New IT Platform — a strategic roadmap that helps organizations transform their IT organizations to meet the needs of the business in a digital age. An underlying principle is greater collaboration between IT and the business to harness technologies to drive business value and increase competitive advantage.

The Integration Fabric connects all applications, services, data, processes, devices and user experience within a single, dynamic web-enabled architecture. Business users can now integrate data sources and processes without custom coding those integration points—giving business users more independence with less dependence on IT.

The Integration Fabric framework includes a set of pre-approved connectors and reusable assets—available through an intuitive interface—along with how-to-guides, leading practices, and intelligent search capabilities. An organization’s ecosystem becomes a “virtual web” of linked resources—bringing together the right information, capabilities and rules at the right time.

Our ongoing research with technology providers like SnapLogic is helping us deliver these next generation integration solutions for enterprise companies.

SnapLogic: How did SnapLogic and PwC start working together? Can you give an example from current projects?
A SnapLogic investor introduced us, and after a briefing by SnapLogic, we saw an immediate opportunity to work together. SnapLogic’s platform is designed to encapsulate leading practices, which is a big part of our New IT Platform approach.

We help many large organizations improve their operations and achieve competitive advantage. Tools like SnapLogic are key enablers in those transformations. Importantly, there is no one size fits all for integration. When we kick off an engagement, we first understand the organization’s strategic business goals—and then help them design a new architecture and integration platform that can enable rapid and seamless integration with the future state architecture in mind.

PWC New IT PlatformSnapLogic: What do CIOs think about this so far?
We’ve had tremendous feedback on New IT Platform and the Integration Fabric from CIOs, as it’s clear that traditional IT models need to evolve—and that integration needs are front and center. Everyone talks about the importance of agility, but integration complexities often stand in the way.
Traditional IT departments have typically used the linear “plan, build, run” approach. With connectors such as Snaps, business users can assemble the components that they need through a library of pre-approved options. Our standardized integration patterns can provide significant flexibility—while also enabling organizations to maintain control. This is an exciting prospect and CIOs are taking notice.

SnapLogic: What are some of the top challenges for companies in moving to this kind of a platform?
Moving to the New IT platform can help organizations achieve competitive advantage in a complex and competitive global market. But getting there requires a mindset shift across the enterprise.

To achieve seamless integration across the ecosystem, IT and the business need to closely collaborate, which has not always been easy. Business and IT need to work hand-in-hand so that business units don’t go around IT – and IT is viewed as a strategic partner that helps the business leverage and integrate digital technologies for competitive advantage. Circumventing IT can ultimately result in higher costs, greater security threats and less productivity.

Also, as a first step, organizations need to think about their integration goals. A holistic Integration Fabric architecture blueprint and plan can help ensure that the team looks beyond what’s required for a single project. Otherwise the effort may compound the larger enterprise integration challenge.
But it’s more than just the technology. It is also important to address the operational aspects and ensure that the right organization and controls are in place to steward the Integration Fabric and ensure that the business value is sustained over time.

SnapLogic: Is this an opportunity for today’s CIO?
Yes. Today’s organizations are no longer isolated enterprises—rather they are diverse and distributed ecosystems of employees, customers, partners and suppliers with different processes and systems. The area of integration presents an opportunity for CIOs to transform how they are viewed and the value they bring to the business. CIOs have the opportunity to play a central role in transforming the business and fostering integration across the ecosystem.

 

SnapIn_HexagonsThis month, we are planning the delivery of our latest Snap Release.

New Snaps

NetSuite Update will be added to the NetSuite Snap Pack. This Snap provides the ability to update the records of an object in NetSuite.

The SAP HANA Snap Pack will be expanded with the addition of a Stored Procedure Snap. Use this Snap to execute a stored procedure in the database and writes any out put to the output view.

This release also introduces the Splunk Search Snap, which executes a search query using Splunk’s REST API.

Updated Snaps

As with all releases, we make continual improvements on our Snaps. Changes in this release focus on Database Snaps, NetSuite Snap Pack, Script Snap, and others. See the Release Notes for more information.