Disconnected data is a drag on innovation

By Scott Behles

What do you consider to be a business’ most valuable asset? Is it the cash it holds? Product inventory? Property perhaps? In the pre-internet age, these traditional assets may have supported businesses and could be easily accounted for on an organization’s balance sheet, but the lifeblood of the 21st-century organization is, without question, data.

Whether it’s customer data, financial data, or increasingly machine data, the insights that can be gleaned from an organization’s data repository are invaluable in developing new products and services, deciding the future roadmap for a business, and gaining competitive advantage.

But are businesses taking full advantage of the data at their fingertips? Particularly in larger enterprises with multiple departments, global offices, and disparate IT systems, data often remains relegated to the department that is considered its primary owner. The finance department handles the accounting data while customer data stays with the marketing and sales teams, for instance.

It’s an antiquated way of handling things, and one that means company leaders and other business decision makers rarely see the full picture of what’s going on across the organization, leading to stifled innovation, unforeseen market threats, and missed opportunities.

Convincing business leaders that this is a serious problem can be a tough sell though. Unless you can assign a dollar figure to how significantly disconnected data is negatively impacting a business, you’ll likely not get much of a reaction.

Thankfully, our new Disconnected Data research has done just that.

We surveyed 500 businesses users and IT decision makers in large businesses across the US and UK and found that the wasted time and resources, duplication of work, and missed opportunities caused by disconnected data is collectively costing businesses $140 billion annually.

That stat alone might raise eyebrows, but when we dug a little deeper we uncovered that this issue in large businesses is likely having a far greater impact.

First, more than one-fifth were unaware of what data other departments actually held and one in six didn’t even know how many data sources actually existed. Against this backdrop, it’s even more surprising to learn that, on average, workers were spending more time searching for, acquiring, entering, or moving data than actually analyzing and making decisions on the data. Workers spending most of their time collecting some but not all data, and at the expense of possibly not incorporating it into their decision-making, paints less than a rosy picture for large businesses’ data-driven strategies.

To their credit, most of our respondents are aware of this problem. More than half (57%) admitted that their organization is struggling with data silos and nearly the same percentage said that data silos are a barrier to meeting their organization’s business objectives.

The business objectives most affected? Seizing new opportunities and driving innovation. A shocking 72% felt that siloed data was causing their business to miss out on opportunities, and a third stated that it was holding back innovation in product and services.

For us here at SnapLogic, that last stat is the real stinger. We firmly believe that innovation should be priority #1 for any business that wants to succeed and thrive in today’s fast-moving digital era. Without innovation, products and services won’t evolve which means customers won’t benefit from the latest developments and will start to look elsewhere. If a business can’t innovate, then its days are numbered. If disconnected data is standing in the way of that innovation, it’s a problem that must be solved. And quickly.

To read our complete study on “The High Cost of Disconnected Data,” to get all the details.

Scott Behles is Head of Corporate Communications at SnapLogic. Follow him on Twitter @sbehles

SnapLogic recognized by Mogul as top workplace for millennial women

By Laura Selig

At SnapLogic, we know that a diverse workplace powers innovation and drives growth. Only by bringing in people with different backgrounds, experiences, and perspectives can we truly build an innovative company for the ages.

So we were thrilled to learn that SnapLogic has been recognized as a top employer for millennial women by Mogul, a leading technology platform and recruiting site that enables professional women around the world to connect, share information and knowledge, and network. We’re proud to report that SnapLogic was ranked #21 on Mogul’s list of the Top 100 Companies For Millennial Women in 2017.

Based on two years of interviews, surveys, and research, Mogul set out to identify the top 100 companies worldwide that are actively leading initiatives to achieve gender equity in the workplace. Mogul notes, with millennials increasingly joining and impacting the workforce, these 100 companies “collectively have the opportunity to shape workplace innovation and accelerate cultural transformation.”

At SnapLogic, more than 30 percent of our global employees in engineering or technical roles are women – higher than the industry average – and we’re working hard to drive this number even higher by fostering a work environment that is focused on inclusiveness and continuous career development. Additionally, SnapLogic’s affinity group, Women@SnapLogic, provides an internal networking group for women to build community and discuss topics relevant to women in the workplace. Female engineers at SnapLogic are working on breakthrough projects in cloud computing, big data, artificial intelligence, Internet of Things, and more, helping to drive our innovation agenda forward.

Are you passionate about technology and looking to put your skills and experience to work at a dynamic, innovative enterprise software upstart? Be sure to check out SnapLogic’s current job openings.

Laura Selig is Vice President, People at SnapLogic.


Data integration: The key to operationalizing innovation

md_craig-BW-1443725112By Craig Stewart

It’s not just a tongue twister. Operationalizing innovation has proven to be one of the most elusive management objectives of the new millennium. Consider this sound bite from an executive who’d just participated in an innovation conference in 2005:

The real discussion at the meeting was about … how to operationalize innovation. All roads of discussion led back to that place. How do you make your company into a systemic innovator? There is no common denominator out there, no shared understanding on how to do that.[1]

The good news is that, in the 12 years since, cloud computing has exploded, and a common denominator clearly emerged: data. Specifically, putting the power of data – big data, enterprise data, and data from external sources – and analytics into users’ hands. More good news: An entirely new class of big data analysis tools[2] has emerged that allows business users to become “citizen data analysts.”

The bad news: There hasn’t been a fast, easy way to perform the necessary integrations between data sources, in the cloud – an essential first step that is the foundation of citizen data analytics, today’s hottest source of innovation.

Until now.

The SnapLogic Enterprise Integration Cloud is a mature, full-featured Integration Platform-as-a-Service (iPaaS) built in the cloud, for the cloud. Through its visual, automated approach to integration, the SnapLogic Enterprise Integration Cloud uniquely empowers both business and IT users, accelerating analytics initiatives on Amazon Redshift and other cloud data warehouses.

Unlike on-premises ETL or immature cloud tools, SnapLogic combines ease of use, streaming scalability, on-premises and cloud integration, and managed connectors called Snaps. Together, these capabilities present a 10x improvement over legacy ETL solutions like Informatica or other “cloud-washed” solutions originally designed for on-premises use, accelerating integrations from months to days.

By enabling “citizen integrators” to more quickly build, deploy and efficiently manage multiple high-volume, data-intensive integration projects, SnapLogic uniquely delivers:

  • Ease of use for business and IT users through a graphical approach to integration
  • A solution built for scale, offering bulk data movement and streaming data integration
  • Ideal capabilities for hybrid environments, with over 400 Snaps to handle relational, document, unstructured, and legacy data sources
  • Cloud data warehouse-readiness with native support for Amazon Redshift and other popular cloud data warehouses
  • Built-in data governance* by synchronizing data in Redshift at any time interval desired, from real-time to overnight batch.

* Why data governance matters

Analytics performed on top of incorrect data yield incorrect results – a detriment, certainly, in the quest to operationalize innovation. Data governance is a significant topic, and a major concern of IT organizations charged with maintaining the consistency of data routinely accessed by citizen data scientist and citizen integrator populations. Gartner estimates that only 10% of self-service BI initiatives are governed[3] to prevent inconsistencies that adversely affect the business.

Data discovery initiatives using desktop analytics tools risk creating inconsistent silos of data. Cloud data warehouses afford increased governance and data centralization. SnapLogic helps to ensure strong data governance by replicating source tables into Redshift clusters, where the data can be periodically synchronized at any time interval desired, from real-time to overnight batch. In this way, data drift is eliminated, allowing all users who access data, whether in Redshift or other enterprise systems, to be confident in its accuracy.

To find out more about how SnapLogic empowers citizen data scientists, and how a global pharmaceutical company is using SnapLogic to operationalize innovation, get the white paper, “Igniting discovery: How built-for-the-cloud data integration kicks Amazon Redshift into high gear.

Craig Stewart is Vice President, Product Management at SnapLogic.

[1] “Operationalizing Innovation–THE hot topic,” Bruce Nussbaum, Bloomberg, September 28, 2005. https://www.bloomberg.com/news/articles/2005-09-28/operationalizing-innovation-the-hot-topic

[2] “The 18 Best Analytics Tools Every Business Manager Should Know,” Bernard Marr, Forbes, February 4, 2016. https://www.forbes.com/sites/bernardmarr/2016/02/04/the-18-best-analytics-tools-every-business-manager-should-know/#825e6115d397

[3] “Predicts 2017: Analytics Strategy and Technology,” Kurt Schlegel, et. al., Gartner, November 30, 2016. ID: G00316349

Why Integration Heritage Matters in the Cloud

“Putting a JSON format on a traditional, XML-based ESB is like making a silk purse out of a sow’s ear.”

– Loraine Lawson’s analogy in reference to the article: Why Buses Don’t Fly in the Cloud: Thoughts on ESBs

I recently wrote about  why the legacy enterprise service bus (ESB) won’t fly in the cloud. Loraine Lawson at IT BusinessEdge reviewed the article and asked the question: Does Integration’s Heritage Matter in the Cloud? At SnapLogic we believe so strongly that heritage matters that we rebuilt our elastic integration platform from the ground up to be fast, multi-point and modern. Here’s why we believe that the heritage of integration products matters:

  1. Because the Innovator’s Dilemma creates major hurdles for legacy integration vendors  to venture completely into this new area of social, mobile, analytics, cloud and (Internet of) Things, which we’re calling SMACT.
  2. Attempts to build on their past successes results in experiments and half-baked solutions.

You’d be surprised how many on-premise integration product managers in Silicon Valley spend more time worrying about earnings per share (EPS) threats than the future of the ESB!

So let’s look at the two reasons why integration heritage matters.

The Innovator’s Dilemma Challenge
Clayton Christensen’s words echo throughout the boardrooms of Silicon Valley today as we face so many technology innovations and disruptions to traditional markets. It is extremely difficult to give up on the gravy train that is the perpetual licensing model of software maintenance. Transitioning to a subscription pricing model, let alone the cultural changes that the software as a service (SaaS) demands, is no simple option. Even if the company’s executives are willing to make this transition, it’s the shareholders that would be very unhappy going from 30-40% operating margins down to single digits. If you were a fly on the wall in the executive boardroom of a company with on-premise heritage trying to enter the cloud market, “cannibalization” will be the most commonly heard word. And even if the board and executives get it, good luck telling the legacy product teams that their baby is no longer beautiful and the sales team that you’re going to introduce a cloud service that will no longer require the on-premise boat anchor up-front price tag.

Half Baked “Hybrid” Integration Solutions
The other reason why on-premise software companies struggle to escape from their heritage is because most meaningful technological innovations cannot be applied as easily as the proverbial “lipstick on the pig”; unless the new offering is completely redesigned and developed from scratch to the latest market requirements and specifications. Not many successful companies have the appetite to do a complete rewrite for the reasons mentioned in the above “Innovator’s Dilemma” section. To draw an analogy, it is like an internal combustion engine-based car manufacturing company making cosmetic changes to their gas combustion-based car and expecting to compete with a state-of-the-art car like Tesla in the electric car market. Nissan had to build its Leaf car from scratch to cater to the electric car market, by building a completely new transmission, a new engine and a new power supply.

Coming back to the specifics of the integration market and why vendor heritage matters, here are some technical reasons why:

  1. Resiliency in the context of integration is the ability of  integration flows to handle changes and variations in data structures without breaking down. Most legacy integration products are strongly-typed or tightly-coupled. In other words, that means the platform needs to know the exact specifications of data that it needs to process when executing the flows. Unfortunately, the SMACT world is not as compliant as we would like. Changes in schemas and data structures are commonplace. Addition of columns to database tables, or a partner accidentally adding additional data fields in a document that gets sent to you should not bring your integrations, and thereby your business, to its knees. Resiliency or a weakly-typed/loosely-coupled paradigm is not something that can be introduced into a product as an afterthought. Introducing resiliency is as involved a process as replacing the transmission of the car that is necessary to move from an IC engine to an electric car. The platform has to be architected on such modern principles from the design phase. Hence, integration heritage does matter.

  2. Legacy integration products which came from the extract, transform and load (ETL) roots were optimized for relational data use cases such as moving large volumes of data from relational data sources into relational data warehouses. These products were built to read rows and columns, operate on them and write rows and columns. These products struggle today when it comes to handling hierarchical data. Similarly, the enterprise application integration (EAI) tools were built for message-oriented integrations that can handle hierarchical data but are optimized for real-time integrations to handle one message at a time in as efficiently as possible. Shedding this heritage to handle broader use cases is no small feat. It’s like changing your car’s engine to be battery powered. Anyone who has had engine trouble know that mechanics recommend buying a brand new car rather than replacing it!

  3. Lastly, integration products with an on-premise heritage are built with the on-premise mindset. Configurations and product libraries are laid out locally on every physical server. These local assets need manual attention when it comes to product upgrades and patch fixes. Managing these local files, especially in a highly distributed environment, turns into a nightmare very fast. This is another of those heritage inheritances that cannot be wished away without a complete product redesign. Think of this lifecycle management of the heritage platform as an oil change that you frequently have to do with your IC engine. Like most people, you as the IC engine car owner needs to take time off your busy schedule and take the car to the shop for minor and major oil changes. Teslas need no oil changes and all product maintenance is software defined. All upgrades are downloaded automatically to the car over mobile network and customers experience no downtime.

In summary, heritage is more of a disadvantage in the rapidly shifting sands of technology innovation. Technology paradigm shifts are still of large magnitude and often demand a new approach and redesign of products and technologies. In this article, we drew an analogy between integration platforms with heritage and IC engines, and between modern integration platforms and electric cars such as Teslas. Of course, one can always rightly argue that at the end of day, both cars will get you to your destination. But, as they say, it’s not the destination but the quality of the journey that makes the destination worth it. And with a modern integration platform as a service (iPaaS), your journey is speedier, more cost-effective and with fewer forced downtimes, making it truly enjoyable.

Next Steps:

  • Read Greg Benson’s posts about the SnapLogic architecture and platform services.
  • Check out some of our resources to learn more about the SnapLogic Integration Cloud.


iPaaS in the News: @SnapLogic Named an EMA Vendor to Watch


Today we announced that Enterprise Management Research has named SnapLogic as “a Vendor to Watch.” The report is written by Evan Quinn, Information Management research director at EMA who has over 20 years of experience as an IT practitioner and nearly 20 years of experience as an industry analyst. He notes in the report:

“As more companies deploy more cloud-based enterprise applications, the need for faster and more agile approaches to data and application integration are becoming paramount. SnapLogic is providing an easy and fast, yet highly dependable, way to integrate the proliferation of SaaS applications with one another and with on-premise applications. SnapLogic offers CIOs a straightforward path to deal with today’s increasingly complicated integration landscape.”

Here are a few other highlights from the report:

  • No matter the business driving force behind application development, from a SaaS vendor adding payment features, to connecting into new supply chain partners through a PaaS, to performing application modernization where legacy app data surfaces in a social context, CIOs and business managers in organizations of all sizes are looking for integration speed without sacrificing dependability.
  • The part that customers delight in when they dig deeper is that SnapLogic combines the speed and developer-friendly approach with enterprise-grade, yet flexible, options.
  • Enterprises have always been open to new, more efficient approaches, as long as they didn’t put the enterprise at risk. It is precisely the balance of new style development with old style dependability that truly sets SnapLogic apart.

You can read the full report on the SnapLogic website here.