G+_PostIcon_1200x627 (1)Big data is evolving as a practice and we are quickly approaching a point at which data will be treated as a single source, which will require a different type of architecture. According to John Myers of Enterprise Management Associates (EMA), this architecture will need to be one that is focused beyond a single platform, where operational and analytical workloads work together. This architecture is called a Hybrid Data Ecosystem.

Join us on Wednesday, April 29th for a live webinar with John, Managing Research Director for EMA’s Business Intelligence practice. This webinar will review the drivers associated with big data implementations, evolving technical requirements for big data environments, and how a robust information management layer is important to big data projects.

During the webinar, we’ll also review how recent EMA research describes the following:

  • Use cases that drive big data and the importance of Internet of Things and streaming applications in big data
  • The impact of cloud implementation avenues for big data projects
  • How the EMA Hybrid Data Ecosystem Information Management Layer coordinates integration between disparate platforms and data types

Register now and join John Myers and the SnapLogic team for this exciting webinar to learn about what constitutes the Hybrid Data Ecosystem – and why it’s a necessity for modern data integration.

Gartner-logoGartner has recently published new research outlining why enterprise IT organizations should begin to converge their application and data integration. According to their research:

“Unnecessary segregated application and data integration efforts lead to counterproductive practices and escalating deployment costs. Integration leaders can use five key drivers to engage stakeholders of data and application integration disciplines to articulate, intersect and work together.”

Written by Eric Thoo and Keith Guttridge, the March 2015 research note provides practical suggestions for enterprise IT organizations who are trying to move at the speed of business and ensure they’re putting in place holistic strategies so their app and data integration technology is an on-ramp to the cloud and big data and not a barrier to adoption.

The research is based upon 2014 international survey data as well as client inquiries during the past 12 months and interactions with attendees at Gartner’s Application Architecture, Development and Integration and Business Intelligence and Information Management Summits.

You can download the Gartner research here.

business-insider-logoThis week SnapLogic co-founder and CEO, Gaurav Dhillon was featured in a profile on Business Insider. He shared his perspective on the news of Informatica, the company he co-founded in the ’90s and ran for 12 years through a successful IPO, being acquired by a private equity firm. He had this to say:

“I genuinely feel that this kind of financial engineering is not good for customers. But from a SnapLogic perspective, we’re cleared for takeoff.”

On Sramana Mitra’s popular One Million by One Million Blog, Gaurav went into more detail about the state of the data and application integration market, Informatica, SnapLogic, and how to build a great company. I’ve embedded the video interview below:

The other shoe dropped today. Last year Tibco was taken over by a private equity firm for $4.3 billion. Today it was announced that Informatica will be taken private in a deal worth $5.3 billion.

So in a matter of months, we’ve seen the market impact of the enterprise IT shift away from legacy data management and integration technologies when it comes to dealing with today’s social, mobile, analytics/big data, cloud computing and the Internet of Things (SMACT) applications and data.

No ESB No ETL. No Kidding!

Last year I posted 10 reasons why old ETL and ESB technologies will continue to struggle in the SMACT era. They are:

  1. Cannibalization of the Core On-Premises Business
  2. Heritage Matters in the Cloud
  3. EAI without the ESB
  4. Beyond ETL
  5. Point to Point Misses the Point
  6. Franken-tegration
  7. Big Data Integration is not Core…or Cloud
  8. Elastic Scale Out
  9. An On-Ramp to On-Prem
  10. Focus and DNA

Last week, SnapLogic’s head of product management wrote the post iPaaS: A new approach to cloud integration. He had this to say about ETL or batch-only data integration:No ETL

“ETL is typically used for getting data in and out of a repository (data mart, data warehouse) for analytical purposes, and often addresses data cleansing and quality as well as master data management (MDM) requirements. With the onset of Hadoop to cost-effectively address the collection and storage of structured and unstructured data, however, the relevance of traditional rows-and-columns-centric ETL approaches is now in question.

Industry analyst and practitioner David Linthicum goes further in his whitepaper: The Death of Traditional Data Integration. The paper covers the rise of services and streaming technologies and when it comes to ETL he notes:

“Traditional ETL tools only focused on data that had to be copied and changed. Emerging data systems approach the use of large amounts of data by largely leaving data in place, and instead accessing and transforming data where it sits, no matter if it maintains a structure or not, and no matter where it’s located, such as in private clouds, public clouds, or traditional systems. JSON, a lightweight data interchange format, is emerging as the common approach that will allow data integration technology to handle tabular, unstructured, and hierarchical data at the same time. As we progress, the role of JSON will become even more strategic to emerging data integration approaches.

And finally, Gaurav Dhillon, SnapLogic’s co-founder and CEO who also co-founded and ran Informatica for 12+ years, had this to say recently about the old way versus the new way of ensuring enterprise data and applications stay connected:

“We think it’s false reasoning to say, ‘You have to pick an ESB for this, and ETL for that.’ We believe this is really a ‘connective tissue’ problem. You shouldn’t have to change the data load, or use multiple integration platforms, if you are connecting SaaS apps, or if you are connecting an analytics subsystem. It’s just data momentum. You have larger, massive data containers, sometimes moving more slowly into the data lake. In the cloud connection scenario, you have lots of small containers coming in very quickly. The right product should let you do both. That’s where iPaaS comes in.”

At SnapLogic, we’re focused on delivering a unified, productive, modern and connected platform for companies to address their Integrator’s Dilemma and connect faster, because bulldozers and buses aren’t going to fly in the SMACT era.

I’d like to wish all of my former colleagues at Informatica well as they go through the private equity process. (Please note that SnapLogic is hiring.)

SNAP_IN_BIG_DATAWhat happens when you’re faced with the challenge of maintaining a legacy data warehouse while dealing with ever-increasing volumes, varieties and velocities of data?

While powerhouses in the days of structured data, legacy data warehouses commonly consisted of RDMS technologies from the likes of Oracle, IBM, Microsoft, and Teradata. Data was extracted, transformed and loaded into data marts or the enterprise data warehouse with traditional ETL tools, built to handle batch-oriented use cases, running on expensive, multi-core servers. In the era of self-service and big data, there’s a rethinking of these technologies and approaches going on in enterprise IT.

In response to these challenges, companies are steadfastly moving to modern data management solutions that consist of NoSQL databases like MongoDB, Hadoop distributions from vendors like Cloudera and Hortonworks, cloud-based systems like Amazon Redshift, and data visualization from Tableau and others. Along this big data and cloud data warehouse journey, many people I speak with have realized that it’s vital to not only modernize their data warehouse implementations, but to future-proof how they collect and drive data into their new analytics infrastructures – requiring an agile, multi-point data integration solution that is both seamless and capable of dealing with structured and unstructured streaming real-time and batch-oriented data. As companies reposition IT and the analytics infrastructure from back-end infrastructure management cost centers to an end-to-end partner of the business, service models become an integral part of both the IT and business roadmaps.

The Flexibility, Power and Agility Required for the New Data Warehouse

In most enterprises today, IT’s focus is moving away from using its valuable resources for undifferentiated-heavy lifting and more into delivering differentiating business value. By using SnapLogic to move big data integration management closer to the edge, resources can be freed up for more value-based projects and tasks while streamlining and accelerating the entire data-to-insights process. SnapLogic’s drag-and-drop data pipeline builder and streaming integration platform removes the burden and complexity of data ingestion into systems like Hadoop, transforming data integration from a rigid, time-consuming process into a process that is more end-user managed and controlled.

Productivity Gains with Faster Data Integration

A faster approach to data integration not only boosts productivity, but in many cases results in substantive cost savings. In one case, a SnapLogic customer with over 200 integration interfaces, managed and supported by a team of 12, was able to reduce their integration management footprint down to less than 2 FTEs, realizing an annual hard cost savings of more than 62% annually, 8:1 annual FTE improvement, infrastructure savings of over 50% and a 35% improvement in their dev-ops release schedule. Ancillary effects with that same customer realized a net productivity gain and increased speed to market by transferring ownership of their Hadoop data ingest process to data scientists and marketers. This shift resulted in the company being more responsive while significantly streamlining their entire data insights process for faster, cheaper and better decision-making across their entire enterprise.

More Agile Company

Increased integration agility means having the ability to make faster, better and cheaper moves and changes. SnapLogic’s modular design allows data scientists and marketers to be light on their feet, making adds, moves and changes in a snap with the assurance they require as new ideas arise and new flavors of data sources enter the picture.

By integrating with Hadoop through the SnapLogic Enterprise Integration Platform, with fast, modern and multi-point data integration, customers have the ability to seamlessly connect to and stream data from virtually any endpoint, whether cloud-based, ground-based, legacy, structured or unstructured. In addition, by simplifying the data integration, SnapLogic customers no longer need to use valuable IT resources to manage and maintain data pipelines, freeing them to contribute to areas of more business value.

Randy Hamilton is a Silicon Valley entrepreneur and technologist who writes periodically about industry related topics including the cloud, big data and IoT.  Randy has held positions as Instructor (Open Distributed Systems) at UC Santa Cruz and has enjoyed positions at Basho (Riak NoSQL database), Sun Microsystems, and Outlook Ventures, as well as being one of the founding members and VP Engineering at Match.com.

Next Steps:

BigData_webinar_GraphicIn February we attended one of the most important big data events of the year – Strata + Hadoop World. In addition to some great conversations with attendees and customers, SnapLogic Chief Scientist Greg Benson also had the opportunity to talk to big data experts, data scientists and other enterprise IT leaders about the data lake and how SnapLogic comes into play with Hadoop-scale integration. In the presentation he covered:

  • SnapLogic’s vision of a unified integration platform as a service (iPaaS)
  • The SnapLogic Designer for building big data integrations
  • SnapLogic key technologies including Hadooplex, SnapReduce and SnapSpark
  • Elastic integration at Hadoop-scale
  • The data lake as a replacement for the enterprise data warehouse

Check out Greg’s recorded presentation below to learn more about how SnapLogic helps customers adopt Hadoop and automate data integration workflows. The full set of slides can also be found here for reference.

For additional information on the new world of big data – and the changing nature of data integration – take a look at this recent eBook from industry expert and analyst Mark Smith of Ventana Research: Attaining Excellence in Big Data Integration.

SnapLogic iPaaSWith Gartner’s 2nd Enterprise Integration Platform as a Service (iPaaS) report published last week, Loraine Lawson at ITBusinessEdge has referred to the iPaaS market as a “Volatile Boom Town.” At SnapLogic, we’re proud to announce that we have once again been recognized as a Visionary Leader in this high-growth market. Here’s what our founder and CEO, Gaurav Dhillon, had to say about the report:

“Since launching the SnapLogic Elastic Integration Platform in late 2013 our product development team has been laser-focused on adding new functionality that solves the challenges of integrating today’s hybrid environments with on-premise, cloud and big data applications. We believe our visionary position is a reflection of growing enterprise customer adoption an continued innovation in delivering a unified platform that addresses both application and big data integration use cases simultaneously.”

Gaurav also talked about how SnapLogic’s cloud integration bet paid off in this diginomica featured interview last week.

At SnapLogic, we’ve set up to address the Integrator’s Dilemma, by pushing for the convergence of application and data integration in the modern era. It’s time to rethink legacy ESB and ETL in the enterprise. Here are some of the companies highlights in the last 12 months. Thanks to all of our customers and partners for their continued support.

Company Execution:

Platform Innovations

  • 300+ Snaps: pre-built connectors, called Snaps, to the most popular cloud and big data systems are now available, including Box, Cloudera, Concur, Eloqua, Hortonworks, NetSuite, MongoDB, Salesforce, SAP, ServiceNow and Workday
  • Snap Patterns: reusable process and data integration patterns make it easy to configure and address the most common integration use cases and data flows for connecting cloud services like Amazon Redshift from Amazon Web Services
  • SnapReduce 2.0: translates data flow pipelines into MapReduce code, enabling customers to ingest, prepare and deliver big data to power next generation analytics
  • Hadooplex: SnapLogic’s hybrid execution grid can run in the cloud, on-premise or as a native YARN application in a Hadoop cluster
  • Ultra Pipelines: dramatically speeds the process by which time-sensitive data is delivered to cloud and big data applications

Partnership Momentum

  • Cloudera and Hortonworks certifications: the Hadooplex introduced to run as a Hadoop 2.0 YARN application
  • SAP Netweaver certification: enables customers to easily share data between their SAP software and other enterprise and cloud applications. SAP Hana Snaps also introduced
  • Integration with Okta: provides seamless, secure access to the SnapLogic platform and a single point of control for user authentication and authorization through the Okta Application Networks
  • AWS Redshift Free Trial and Snap Patterns: quickly build, deploy and efficiently manage multiple high-volume, data-intensive integration tasks delivering data to and from a cloud data warehouse
  • Salesforce Analytics Cloud Snaps and Patterns: powering the Wave with easy data loading and transformation

As you can see it was a busy year for the SnapLogic team…and we’re just getting started! This week we’ll be at Gartner BI in Las Vegas and on April 9th, we’ll be at the AWS Summit in San Francisco. You can check out all of our upcoming events here and be sure to Contact Us if you’d like more information on the SnapLogic Elastic Integration Platform.