Igniting data discovery tools: SnapLogic delivers a “quanta” in improvement over Informatica

md_craig-BW-1443725112In my previous blog post, I talked about how a pharmaceutical company uses SnapLogic and Amazon Redshift to capitalize on market and environmental fluctuations, driving sales for its asthma relief medication. In this post, I’ll tell you the path the company took to get there. Hint: It wasn’t a straight one.

An IT organization abandons Informatica

Several months prior to launching its current environment, with data flows powered by SnapLogic, the pharmaceutical company tried, unsuccessfully, to integrate its data warehouses in Redshift using Informatica PowerCenter and Informatica Cloud. The IT team’s original plan was to move data from Salesforce, Veeva, and third-party sources into Amazon Simple Storage Service (S3), and then integrate the data into Redshift for sales and marketing analytics.

However, the project stalled due to difficulty with Informatica PowerCenter, the IT team’s initial choice for data integration. PowerCenter, which Informatica describes as a “metadata-driven integration platform,” is a data extract, transfer, and load (ETL) product rooted in mid-1990s enterprise architecture. The team found PowerCenter complicated to use and slow to deliver the urgently needed data integration technologies.

Looking for faster results, the pharmaceutical company then attempted to use Informatica Cloud, Informatica’s cloud-based integration solution. The data integration initiative was again derailed, this time by the solution’s lack of maturity and functionality. The pharmaceutical company’s data was forced back on-premises, jeopardizing the entire cloud data warehouse initiative.

Data integration aligned with the cloud

But the IT team kept searching for the right data integration solution. “Cloud was instrumental to our plans, and we needed data integration that aligned with where we were headed,” said the senior business capability manager in charge of the integration project. The pharmaceutical company chose the SnapLogic Enterprise Integration Cloud.

After a self-evaluation, the IT team was able to quickly build data integrations with SnapLogic; no specialized resources or consultants were required. To accomplish the integrations in Redshift, the pharmaceutical company used:

  • Salesforce Snap
  • Redshift Snap
  • Various RDBMS Snaps
  • ReST/SOAP Snaps
  • Transformation Snaps

With the data integration accomplished in a matter of days, the IT organization was assured that current skills sets could support the company’s future global BI architecture. In addition, the IT team found the SnapLogic Enterprise Integration Cloud easy enough for business users, such as the marketing team, to integrate new data into Redshift.

Leveraging Redshift’s nearly infinite availability of low-cost data storage and compute resources, the analytic possibilities are equally limitless – igniting the marketing team’s discovery of new strategies to drive new insights, revenues, and operational efficiencies.

SnapLogic delivers a “quanta” in improvement 

What is quanta? It’s the plural of the word “quantum,” a physics term that describes “a discrete quantity of energy proportional in magnitude to the frequency of the radiation it represents.” If you’re not a physicist, your closest association is probably “quantum leap” – basically a gigantic leap forward.

Which is exactly what SnapLogic delivers. With regard to Informatica, Gaurav Dhillon, founder and CEO of SnapLogic, says:

“Fundamentally, I believe that SnapLogic is 10 times better than Informatica. That’s a design goal, and it’s also a necessary and sufficient condition for success. If a startup is going to survive, it’s got to have some 10x factor, some quanta of a value proposition.

“The quanta over the state of the art – the best-of-the-best of the incumbents – is vital. SnapLogic can fluently solve enterprise data problems almost as they are happening. That has a ‘wow’ factor people experience when they harness the power of our data integration technology.”

The SnapLogic Enterprise Integration Cloud is a mature, full-featured Integration Platform-as-a-Service (iPaaS) built in the cloud, for the cloud. Through its visual, automated approach to integration, the SnapLogic Enterprise Integration Cloud uniquely empowers both business and IT users, accelerating cloud data warehouse and analytics initiatives on Redshift and other cloud data warehouses

Unlike on-premises ETL or immature cloud tools, SnapLogic combines ease of use, streaming scalability, on-premises, and cloud integration, and managed connectors. Together, these capabilities present an improvement of up to 10 times over legacy ETL solutions such as Informatica or other “cloud-washed” solutions originally designed for on-premises use, accelerating cloud data warehouse integrations from months to days.

To learn more about how SnapLogic allows citizen data scientists to be productive with Amazon Redshift in days, not months, register for the webcast “Supercharge your Cloud Data Warehouse: 7 ways to achieve 10x improvement in speed and ease of Redshift integration.”

Craig Stewart is Vice President, Product Management at SnapLogic.

Discovery in overdrive: SnapLogic and Amazon Redshift power today’s pharma marketing

md_craig-BW-1443725112At its most fundamental, pharmaceutical marketing is based on straightforward market sizing and analytic baselines:

“The global market is composed of many submarkets [aka therapeutic categories] (TCs), whose number is given and equal to nTC. Each TC has a different number of patients (PatTC) in need of treatment for a specific disease, which determines the potential demand for drugs in each submarket. This number is set at the beginning of each simulation by drawing from a normal distribution [PatTC~N(μp,σp)] truncated in 0 to avoid negative values, and it is known by firms. Patients of each TC are grouped according to their willingness to buy drugs characterised by different qualities.”*

Yet capturing market share in today’s competitive environment is anything but easy. In the recent past, an army of sales reps would market directly to doctors, their efforts loosely coupled with consumer advertising placed across demographically compatible digital and traditional media.

This “spray and pray” approach with promotional spending, while extremely common, made it difficult to pinpoint specific tactics that drove individual product revenues. Projections and sales data factored heavily into the campaign planning stage, and in reports that summarized weekly, monthly, and quarterly results, but the insights gleaned were nearly always backward-looking and without a predictive element.

A pharmaceutical company pinpoints opportunity

Today, sophisticated pharmaceutical marketers have a much firmer grasp of how to use data to drive sales in a predictive manner – by deploying resources with pinpoint precision. A case in point: To maximize the market share of a prescription asthma medication, a leading pharmaceutical company uses SnapLogic and Amazon Redshift to analyze and correlate enormous volumes of data on a daily basis, capitalizing on even the smallest market and environmental fluctuations.

  • Each night, the marketing team takes in pharmacy data from around the US to monitor sales in each region, to learn how many units of the asthma medication sold the previous day. These numbers are processed, analyzed, and reported back to the sales team the following morning, allowing reps to closely monitor progress against their sales objectives.
  • With this data, the pharmaceutical marketing team can monitor, at aggregate and territory levels, the gross impact of many variables including:
    • Consumer advertising campaigns
    • Rep incentive programs
    • News coverage of air quality and asthma
  • However, the pharmaceutical marketing team takes its exploration much deeper. Layered on top of the core sales data, the marketing team correlates weather data from the National Weather Service (NWS) and multiple data sets from the US Environmental Protection Agency (EPA), such as current air quality, historic air quality, and air quality over time. Like the sales data, the weather and EPA data cover the entire US.

By correlating these multiple data sets, the marketing team can extract extraordinary insights that improve tactical decisions and inform longer-term strategy. At a very granular, local level, the team can see:

  • How optimal timing and placement of advertising across digital and traditional media drives demand
  • Which regional weather conditions stimulate the most sales in specific locales
  • The impact of rep incentive programs on sales
  • How news coverage of air quality and asthma influences demand

Ultimately, the pharmaceutical marketing team can identify, with uncanny precision, markets to concentrate spending on local and regional media, which can change on a constant basis. In this way, prospective consumers are targeted with laser-like accuracy, raising their awareness of the pharmaceutical company’s asthma medication at the time they need it most.

The results of the targeted marketing strategy are clear: The pharmaceutical company has enjoyed significant market share growth with its asthma relief medication, while reducing advertising costs due to more effective targeting.

Tools to empower business users

The pharmaceutical industry example exemplifies perhaps the biggest data analytics trend in recent business history: massive demand for massive amounts of data, to provide insight and drive informed decision-making. But five years after data scientist was named “the sexiest job of the 21st century,” it’s not data scientists who are gathering, correlating, and analyzing all this data; at the most advanced companies, it’s business users. At the pharmaceutical company and countless others like it, the analytics explosion is ignited by “citizen data scientists” using SnapLogic and Redshift.

In my next blog post, the second of this two-part series, I’ll talk about how SnapLogic turned around a failing initial integration effort at the pharmaceutical company, replacing Informatica PowerCenter and Informatica Cloud.

To find out more on how to use SnapLogic with Amazon Redshift to ignite discovery within your organization, register for the webcast “Supercharge your Cloud Data Warehouse: 7 ways to achieve 10x improvement in speed and ease of Redshift integration.”

Craig Stewart is Vice President, Product Management at SnapLogic.

* JASSS, A Simulation Model of the Evolution of the Pharmaceutical Industry: A History-Friendly Model, October 2013

Winter 2017 Release Is Now Available

As enterprises grow and adopt best of breed solutions based in the cloud, on-premises and/or hybrid, integrating data between varied applications, databases and data warehouses (used by the enterprise) continues to be a challenge. New solutions are rapidly adopted, and technical and non-technical users alike need help to meet the challenge of quickly integrating the data from multiple sources into one view to make decisions at the speed of business.

snp-76209-winterrelease-484x252-facebookThe release includes several new Snaps and Snap updates that make it faster and easier to integrate Workday, NetSuite and Amazon Redshift with other applications and data sources across the enterprise. All three systems are increasingly popular as businesses embrace the cloud to run their business, a “cloud shift” that Gartner says will drive more than $1 trillion in technology spending by 2020.

Here is a brief overview of new and enhanced Snaps:

  • Confluent KafkaThe need for streaming data becomes more important and today about one-third of the Fortune 500 uses Kafka. SnapLogic is pleased to introduce a new Snap for Confluent’s distribution of Apache KafkaTM, an enterprise-ready solution that connects data sources, applications and IoT devices in real time.
  • TeradataSeveral new Snaps have been added to Teradata Snap Pack expanding support with Teradata TPT Load, TPT Update Snap, and Teradata Export to HDFS Snap which allows customers to easily export data from Teradata to an HDFS cluster without the need for any additional installation or complex configuration.
  • Workday: Workday Read Snap has been enhanced to provide a simplified Workday output format making it even easier to be consumed by downstream systems.
  • NetSuite: Asynchronous operations support for NetSuite, enables more efficient use of NetSuite’s capabilities, through new Snaps including Netsuite Async  Upsert, Async Search, Async Delete List, Async GetList, Check Async Status and GetAsync Result Operations Support Snap.
  • Amazon Redshift: Our customers use Redshift to connect multiple on-premises data sources and applications to Redshift without any coding. The Winter 2017 release introduces a new Snap to execute multiple RedShift commands in one Snap, thereby making RedShift data integration pipelines even more easy to create and manage.
  • Amazon S3: The Winter 2017 release brings additional streaming performance improvement while writing to an Amazon S3 bucket.

Continued Enterprise Focus: Introducing Asset Search Functionality

SnapLogic continues to be the best platform for enterprise IT and LOB teams to integrate applications and data sources without any coding. Enterprises often have thousands of pipelines, files and accounts and it’s hard to search for a given asset. The Winter 2017 release allows customers to quickly search for assets and also filter search outputs.

Security and Performance Enhancements

Security and performance continue to be focus areas for SnapLogic. To further tighten user passwords, the Winter 2017 release enforces enhanced password complexity requirements. Customers can also configure session timeout and idle timeout parameters. In addition, the MongoDB snap pack has been extended to support SSL.

SnapLogic is committed to supporting the growing enterprise’s needs. We hope you will find the new Confluence Kafta snap, expanded support for WorkDay, Netsuite, Amazon RedShift, enhanced search and security useful. Customers can start using the capabilities described in the Winter 2017 release right away. For more information on the Winter 2017 release, including demo videos, see www.snaplogic.com/winter2017.

Connect with SnapLogic at AWS re:Invent

This week, the SnapLogic team will be supporting one of our partners, Amazon Web Services, in Las Vegas for the annual AWS re:Invent conference. This gathering of the global AWS community will feature hands-on labs and bootcamps and cover topics such as infrastructure maintenance, and improving developer productivity, network security and application performance.

Continue reading “Connect with SnapLogic at AWS re:Invent”

Don’t Let Cloud Be Another Silo: Accelerate Your AWS data integration

Gone are the days when enterprises had all of their apps and data sources on-premises. Today is the era of big data, cloud and hybrid deployments. More and more enterprises are rapidly adopting different SaaS applications and hosting their solutions in public clouds including Amazon Web Services and Microsoft Azure. But soon enterprises realize that their SaaS applications and on-premises data sources are not integrated with their public cloud footprint and the integration itself becomes an expensive and time consuming undertaking.

Continue reading “Don’t Let Cloud Be Another Silo: Accelerate Your AWS data integration”

SnapLogic Roadshow – Is The Data Warehouse Dead?

Short answer to that provocative question: no – it’s just changing. Our data integration road show hit four U.S. cities over the past two weeks. The keynote presentation was delivered by James Markarian, SnapLogic’s CTO. He shared his perspective on modern data management, the role of the data warehouse in a hybrid cloud environment, and important considerations for an enterprise data lake strategy.

James Markarian presents at the SnapLogic roadshow July 2016.
James Markarian presents at the SnapLogic roadshow July 2016.

 

We were also joined by our partner, Amazon Web Services, who highlighted their cloud data management platform, with an emphasis on Amazon Redshift. Together we highlighted some of the joint SnapLogic/AWS success stories around hybrid cloud data integration, including GameStop, Box, CapitalOne, and eero.

The bulk of James’ keynote focused on the changes to the data landscape that are affecting the role and structure of the data warehouse.  He described “data warehousing 1.0” from the 1980’s which introduced a convenient, single place to warehouse all data, but which was expensive and reliant on scripting to integrate sources. He contrasted that with “data warehousing 2.0” of the 1990’s which saw the rise of ETL processes and data marts, but which was still rigid and typically on-premises. Since that era, however, data warehousing has remained generally static. Now, with the dramatic increase in unstructured/polystructured data, plus cloudification of data sources, data warehousing 2.0 has fallen a bit short. Enter the data lake. James cautioned the audience not to think of a data lake as an amorphous, no-rules dumping ground for unstructured data. Instead, he identified multiple “zones” within the data lake, each of which has certain requirements, rules and uses.

Finally, James illustrated how lakeshore data marts and cloud-based data warehouses – connected using SnapLogic – address some of the risk and high labor costs that can be associated with Hadoop-centric data lakes.

Attendees – some of whom were already far along in their data lake adoption journey and some still in a “data warehouse 2.0” environment – certainly left with food for thought. Watch this space for the next time SnapLogic comes to a city near you.

In the mean time, don’t forget to subscribe to our data management podcast series called SnapTalk.

Is the Data Warehouse Dead?

snaplogic_aws

SnapLogic and Amazon Web Services are hosting a series of exclusive live seminars starting this week in Dallas. Next week we’ll be in Chicago and New York, followed by Palo Alto later in the month. The seminar series is focused on the future of data warehouse solutions and analytics in the modern enterprise. A key question that we’ll address is: Is the Data Warehouse Dead? Continue reading “Is the Data Warehouse Dead?”