Data management takes center stage at Rutberg 2017 conference

Each year, research-centric investment bank Rutberg & Company gathers top business leaders and technology experts for an intimate, two-day forum where they discuss and debate the technology, ideas, and trends driving global business. The annual Rutberg 2017 conference took place last week in Half Moon Bay, California, and data management was front and center.

SnapLogic CEO Gaurav Dhillon joined Mesosphere CEO Florian Leibert and Segment CEO Peter Reinhardt for a spirited panel discussion on the growing data management opportunities and challenges facing enterprises today. The panel was moderated by Fortune reporter Jonathan Vanian.

A number of important data management and integration trends emerged, including:

  • LOB’s influence grows: Gaurav noted that more and more, “innovation is coming from the LOB,” whether in Sales, Marketing, Finance, HR, or elsewhere in the organization. These LOB leaders are tech-savvy, are responsible for their own P&L’s, and they know speed and agility will determine tomorrow’s winners. So they’re constantly on the hunt for the latest tech solutions that will drive innovation, spur growth, and help them beat the competition.
  • Data fragmentation on the rise: With individual LOBs procuring a flurry of new cloud applications and technologies, the result is often business silos and a disconnected enterprise. “The average enterprise has 10x more SaaS apps than a CIO thinks,” said Gaurav of the increasing SaaS sprawl, which is requiring CIOs to think differently about how they integrate and manage disparate apps and data sources across the enterprise.
  • Self-service integration is here to stay: The bigger a company gets – with more apps, more end-points, more data-types, more fragmentation – there’s never going to be enough humans to manage the required integration in a timely manner, explained Gaurav. Enter new, modern, self-service integration platforms. “The holy grail of integration is self-service and ease-of-use … we have to bring integration out of the dungeon and into the light,” Gaurav continued. And this means getting integration into the hands of the LOB, and making it fast and easy. The days of command-and-control by IT are over: “Trying to put the genie back in the bottle is wrong; instead you need to give the LOBs a self-service capability to wire this up on their own,” noted Gaurav.
  • AI will be a game-changer: Artificial intelligence (AI) and machine learning (ML) are already making apps, platforms, and people smarter. Like with Google auto-complete or shopping on Amazon, we’re already becoming accustomed to assistance from, and recommendations by, machines. “Software without AI will be like Microsoft Word or email without spell-check,” it will be jarring not to have it, said Gaurav. AI is already being applied to complex tasks like app and data integration; it’s not a future state, he said, the start of “self-driving integration is happening today.”
  • The enterprise is a retrofit job: For all the latest advances – new cloud apps, AI and ML technologies, self-service integration platforms – the enterprise remains a “retrofit job,” where the new must work with the old. Large, global enterprises aren’t about to throw out decades of technology investment all at once, particularly if it is working just fine or well-suited to handle certain business processes. So, new cloud technologies will need to work with older on-premise solutions, once again cementing integration platforms as a critical piece of an enterprise technology strategy. “It will be a hybrid world for a long, long time,” concluded Gaurav.

Without question, data has become any organization’s most valuable asset, and those that are able to integrate, manage, and analyze data effectively will be the winners of tomorrow.

SnapLogic recognized by Mogul as top workplace for millennial women

By Laura Selig

At SnapLogic, we know that a diverse workplace powers innovation and drives growth. Only by bringing in people with different backgrounds, experiences, and perspectives can we truly build an innovative company for the ages.

So we were thrilled to learn that SnapLogic has been recognized as a top employer for millennial women by Mogul, a leading technology platform and recruiting site that enables professional women around the world to connect, share information and knowledge, and network. We’re proud to report that SnapLogic was ranked #21 on Mogul’s list of the Top 100 Companies For Millennial Women in 2017.

Based on two years of interviews, surveys, and research, Mogul set out to identify the top 100 companies worldwide that are actively leading initiatives to achieve gender equity in the workplace. Mogul notes, with millennials increasingly joining and impacting the workforce, these 100 companies “collectively have the opportunity to shape workplace innovation and accelerate cultural transformation.”

At SnapLogic, more than 30 percent of our global employees in engineering or technical roles are women – higher than the industry average – and we’re working hard to drive this number even higher by fostering a work environment that is focused on inclusiveness and continuous career development. Additionally, SnapLogic’s affinity group, Women@SnapLogic, provides an internal networking group for women to build community and discuss topics relevant to women in the workplace. Female engineers at SnapLogic are working on breakthrough projects in cloud computing, big data, artificial intelligence, Internet of Things, and more, helping to drive our innovation agenda forward.

Are you passionate about technology and looking to put your skills and experience to work at a dynamic, innovative enterprise software upstart? Be sure to check out SnapLogic’s current job openings.

Laura Selig is Vice President, People at SnapLogic.

 

Iris – Can you build an integration pipeline for me?

The promise of Artificial Intelligence technology is flourishing. From Amazon shopping recommendations, Facebook image recognition, and personal assistants like Siri, Cortana, and Alexa,  AI is becoming part of our everyday lives, whether we know it or not. These apps use information collected from your past requests to make predictions and deliver results that are tailored to your preferences. 

The importance of AI in today’s world is not lost upon us at SnapLogic. We are always trying to keep up with the latest innovations and technologies, so making our software fast, efficient, and automated for our customers has always been our goal. With the Spring release, SnapLogic launched the SnapLogic Integration Assistant. The SnapLogic Integration Assistant is a recommendation engine that uses Artificial Intelligence and machine learning to predict the next step in building a data pipeline. Iris uses advanced algorithms to collect information from millions of metadata elements and billions of data flows to make predictions and deliver results that are tailored to the customer’s needs.

Currently, customers build pipelines by searching and selecting from over 400 Snaps in the SnapLogic catalog and dragging and dropping them into the canvas. Repeating this step for every single Snap, although easy, can make a pipeline building process somewhat tedious and time-consuming. But with the Integration Assistant, operations like these are simplified to give business users the right next steps in the building process, making pipeline building easy and efficient. Besides efficiency and speed, the “self-driving” software shortens the learning curve for line-of-business users to manage their data flows while freeing technology staff for higher-value software development. See how it works in this video.

In the next few steps,  learn how to enable this feature and start building interactive pipelines yourself.

Right now, we have two ways of building pipelines:

  • Choose a Snap from the SnapLogic catalog
  • Use the Integration Assistant for recommending the right Snaps

How to enable the Integration Assistant feature

By default, the Integration Assistant option is turned off,  allowing you to continue building pipelines by selecting Snaps in the SnapLogic Catalog. However, to utilize the Integration Assistant, just head to the Settings icon and check the Integration Assistant option.

Once the Integration Assistant is enabled, you’ll immediately see the benefits of the self-guided user interface. Drag the first snap onto the canvas and the Integration Assistant instantly kicks in and highlights the next suitable Snap. At the same time, it also opens up another panel that lists suggested Snaps on the right-hand side of the canvas. These AI-driven Snap recommendations are based on the historical metadata from your previous workflows.

Next, you can choose to click the highlighted Snap or pick from the recommended list by dragging the suitable Snap into the canvas. This process continues further until you select a snap with a closed output. At this point, the Integration Assistant will stop suggesting Snaps and the pipeline will be ready for execution.

As you can see, the Integration Assistant improves your pipeline building experience by suggesting Snaps that are the best fit for your organization based on the historical metadata flows.

Interested in learning more? Watch a quick demo on our YouTube channel – SnapLogic Spring 2017: Integration Assistant.”

Namita Prabhu is Senior QA Manager at SnapLogic.

Spring 2017 Release: Self-driving integration, field cryptography, MS Dynamics 365 CRM and more

Today the Spring 2017 release is available and featuring artificial intelligence (AI) technology that promises to dramatically reduce the time and cost of cloud, analytics, and digital transformation initiatives. The first AI-powered feature is SnapLogic Integration Assistant, a recommendation engine that delivers expert step-by-step guidance to improve the speed and quality of building a data pipeline — with up to 90% accuracy. Integration Assistant is available to all of our customers starting today at no charge.

SnapLogic’s Spring 2017 release also introduces new and enhanced software that helps our customers jumpstart their CRM, HR, and cloud data warehouse software projects. Finally, the release has added platform-wide features designed to save time and increase productivity.

New or enhanced capabilities in the Spring 2017 release include:

  • SnapLogic Integration Assistant: The SnapLogic Integration Assistant is a recommendation engine that uses machine learning to predict the next step in building a data pipeline for the cloud, analytics, and digital initiatives – with up to 90% accuracy. It is part of SnapLogic’s “Iris” technology – an industry first in applying artificial intelligence for enterprise integration. See how it works in this video.
  • Microsoft Dynamics CRM integration: A new Snap Pack is available to help users create, read, and update records in the cloud and on-premises versions of Microsoft Dynamics. Also, users can delete a record based on account ID and also search with various filter options.
  • Workday integration: The Workday Read Snap now supports page number and page size and shows great performance improvements. The Workday Write Snap now supports bulk operations for many objects and has improved performance from hours and minutes to just seconds.
  • Confluent integration: The updated Confluent Acknowledge Snap greatly reduces potential data losses, eliminates duplicates, and externally acknowledges each message.
  • Enhanced Snaps for Amazon Redshift, Anaplan, Tableau, Apache Hive + Kerberos.
  • Field encryption and decryption: The Transform Snap Pack now includes new Snaps to encrypt and decrypt field values for sensitive data, or entire documents, providing a greater level of data security.
  • Platform enhancements: These updates save time and enhance productivity, including Snaplex restart; enhanced Asset Search with Snap labels; network statistics display to aid troubleshooting and performance optimization; and parameterizable accounts that make it easier to automate or dynamically assign environments in a development lifecycle.

SnapLogic is committed to continuous innovation and the features in the Spring 2017 release for the SnapLogic Enterprise Integration Cloud are examples of how we continue to make integrating data warehouses, applications, IoT, and big data streams faster, easier. and more powerful. To learn more, go to www.snaplogic.com/spring2017.

Applying machine learning tools to data integration

greg-bensonBy Gregory D. Benson

Few tasks are more personally rewarding than working with brilliant graduate students on research problems that have practical applications. This is exactly what I get to do as both a Professor of Computer Science at the University of San Francisco and as Chief Scientist at SnapLogic. Each semester, SnapLogic sponsors student research and development projects for USF CS project classes, and I am given the freedom to work with these students on new technology and exploratory projects that we believe will eventually impact the SnapLogic Enterprise Integration Cloud Platform. Iris and the Integration Assistant, which applies machine learning to the creation of data integration pipelines, represents one of these research projects that pushes the boundaries of self-service data integration.

For the past seven years, these research projects have provided SnapLogic Labs with bright minds and at the same time given USF students exposure to problems found in real-world commercial software. I have been able to leverage my past 19 years of research and teaching at USF in parallel and distributed computing to help formulate research areas that enable students to bridge their academic experience with problems found in large-scale software that runs in the cloud. Project successes include Predictive Field Linking, the first SnapLogic MapReduce implementation called SnapReduce, and the Document Model for data integration. It is a mutually beneficial relationship.

During the research phase of Labs projects, the students have access to the SnapLogic engineering team, and can ask questions and get feedback. This collaboration allows the students to ramp up quickly with our codebase and gets the engineering team familiar with the students. Once we have prototyped and demonstrated the potential for a research project we transition the code to production. But the relationship doesn’t end there – students who did the research work are usually hired on to help with transitioning the prototype to production code.

The SnapLogic Philosophy
Iris technology was born to help an increasing number of business users design and implement data integration tasks that previously required extensive programming skills. Most companies must manage an increasing number of data sources and cloud applications as well as an increasing amount of data volume. And it’s data Integration platforms that help business connect and transform all of this disparate data. The SnapLogic philosophy has always been to truly provide self-service integration through visual programming. Iris and the Integration Assistant further advances this philosophy by learning from the successes and failures of thousands of pipelines and billions of executions on the SnapLogic platform.

The Project
Two years ago, I led a project that refined our metadata architecture and last year I proposed a machine learning project for USF students. At the time, I gave some vague ideas about what we could achieve. The plan was to spend the first part of the project doing data science on the SnapLogic metadata to see what patterns we could find and opportunities for applying machine learning.

One of the USF graduate students working on the project, Thanawut “Jump” Anapiriyakul, discovered that we could learn from past pipeline definitions in our metadata to help recommend likely next Snaps during pipeline creation. Jump experimented with several machine learning algorithms to find the ones that give the best recommendation accuracy. We later combined the pipeline definition with Snap execution history to further improve recommendation accuracy. The end result: Pipeline creation is now much faster with the Integration Assistant.

The exciting thing about the Iris technology is that we have created an internal metadata architecture that supports not only the Integration Assistant but also the data science needed to further leverage historical user activity and pipeline executions to power future applications of machine learning in the SnapLogic Enterprise Cloud. In my view, true self-service in data integration will only be possible through the application of machine learning and artificial intelligence as we are doing at SnapLogic.

As for the students who work on SnapLogic projects, most are usually offered internships and many eventually become full-time software engineers at SnapLogic. It is very rewarding to continue to work with my students after they graduate. After ceremonies this May at USF, Jump will join SnapLogic full-time this summer, working with the team on extending Iris and its capabilities.

I look forward to writing more about Iris and our recent technology advances in the weeks to come. In the meantime, you can check out my past posts on JSON-centric iPaaS and Hybrid Batch and Streaming Architecture for Data Integration.

Gregory D. Benson is a Professor in the Department of Computer Science at the University of San Francisco and Chief Scientist at SnapLogic. Follow him on Twitter @gregorydbenson.