New Podcast Series: SnapTalk

We are pleased to announce our new podcast series called SnapTalk. The series will feature short, 10-15 min. episodes on topics relevant to big data, data management and app and data integration. Our host for the series is Ravi Dharnikota, SnapLogic’s head of enterprise architecture. Each episode features a special guest in conversation with Ravi, such as SnapLogic’s chief scientist, Greg Benson.

This project grew out of the great conversations we have at Snappy Hour. Eating lunch as a group at least a couple of times a week and our weekly happy hour (called Snappy Hour) are big parts of the SnapLogic culture. And, invariably, the conversations at these gatherings range from the lightweight, such as the latest episode of Game of Thrones, to the complex, such as the future of Spark and what makes streaming data streaming. This podcast series is intended to capture the essence of those ad hoc discussions, get people thinking, and hopefully inspire additional discussions.

The first episodes are posted now and cover topics such as Spark, streaming data and Kafka. Stay tuned to this space for the next episode. The SnapTalk playlist is here and our new SoundCloud channel is here– I hope you’ll subscribe, and we welcome your feedback.

SnapLogic Best Practices: Deploying Projects Between Phases

[update – check out what’s new in our Spring 2016 release – the Metadata Snaps are also useful for Lifecycle Management requirements]

One of the areas our services team and partners spend time with customers early in a SnapLogic Elastic Integration Platform deployment is on deploying from one project phase to the other (Dev -> QA -> Prod). There are a number of different configuration options. In this post, I’ll describe one. First a few assumptions:

  • The enterprise Lifecycle Management feature is not implemented in this example
  • The phases that are in use are Development, QA and Production
  • Each phase in use is being managed at a project level as a separate project with in a single Organization Setup
  • The users have the necessary permissions to perform the operations described in this post
  • The enhanced account encryption feature is not in use in the current SnapLogic Org

Continue reading “SnapLogic Best Practices: Deploying Projects Between Phases”

Designing Ultra Pipelines: Types of Views

In my first post on SnapLogic Ultra Pipelines, I began to review aspects to consider when designing these low-latency pipelines. Once you’ve determined the right number of views, you need to determine the type of views. The unconnected views in an Ultra Pipeline act as the gatekeepers of the task, receiving and returning documents from the external applications. Continue reading “Designing Ultra Pipelines: Types of Views”

Designing Ultra Pipelines

Ultra Pipeline tasks are used to implement real-time web service integrations which require expected response times to be close to a few sub seconds. In the first series of posts I’ll outline some of the key aspects of designing Ultra Pipelines. In the second series of posts I’ll focus on monitoring these low-latency tasks.

Setting up Ultra Pipelines
Ultra Tasks let a pipeline, known as an Ultra Pipeline, continuously consume documents from external sources.

Because Ultra Pipelines are analogous to web service request/response architecture, the following aspects should be considered in designing Ultra Pipelines in SnapLogic. Continue reading “Designing Ultra Pipelines”

Attention Enterprise Architects: Sage Advice from Jason Bloomberg

bloomberg_rest_seminarIf you’re interested in enterprise IT architecture, chances are you’ve heard of Jason Bloomberg. The president of Intellyx, which is “the first and only industry analysis, advisory, and training firm focused on agile digital transformation,” Jason is a globally recognized expert on agile digital transformation who writes and speaks on how today’s disruptive enterprise technology trends support the digital professional’s business transformation goals. He is a prolific writer who is a regular contributor to Forbes, has a biweekly newsletter called the Cortex, and several contributed blogs. His latest book is The Agile Architecture Revolution (Wiley, 2013).

Recently Jason has published a series of articles that are directed towards today’s enterprise architect (EA), focusing on what’s new and what’s different in the era of social, mobile, analytics, cloud and the Internet of Things (SMACT). Here are the four posts he’s written so far: Continue reading “Attention Enterprise Architects: Sage Advice from Jason Bloomberg”

JSON is the New CSV and Streams are the New Batch

Mark MadsenThis is the 2nd post in the series from Mark Madsen’s whitepaper: Will the Data Lake Drown the Data Warehouse? In the first post,  Mark outlined the differences between the data lake and the traditional data warehouse, concluding: “The core capability of a data lake, and the source of much of its value, is the ability to process arbitrary data.”

In this post, Mark reviews the new environment and new requirements: Continue reading “JSON is the New CSV and Streams are the New Batch”

SnapLogic Tips and Tricks: XML Generator Snap Overview (Part 2)

In the first part of this series, we explained how to use the XML Generator Snap to generate XML based off an XSD. In this post, we will cover how to map to the JSON schema upstream.

Example 2: Mapping to XML Generator via XSD
Lets use a JSON Generator to provide the input order data, such as defined below:

We then map the data using the Mapper Snap, which has access to the XSD of the downstream XML Generator Snap of the previous example (now with an added input view).

xml-gen-3

Here we map the items to the item list on the target. Further we map the city, address, country and name to the shipTo object on the target and then finally we map the name against orderperson and orderId against @orderId on the target. The @ indicates we map against an XML attribute.

Hint: the Mapper Snap was enhanced in the Fall 2014 release to allow viewing the data on the in/output while doing the mappings (on the bottom, expanded with the arrow in the middle)

Lets look at the output of the XML Snap:

xml-gen-4

Here we see that each incoming order document was translated into an XML string. We include the original data from the input view, in case it is further needed downstream.
The XML Generator Snap can validate the generated content if needed using the “Validate XML” property.

In our next post in this series, we will demonstrate how the XML Generator Snap validates the generated XML against the XSD.

Other Resources: