Enterprise IoT: Watching Cat Videos Without Getting Caught (or, How I Learned to Stop Looking Over My Shoulder and Trust the CEO Proximity Alert

We have a slight problem at SnapLogic. While we spend a vanishingly small percent of the day watching adorable cat videos on the Internet, it seems our CEO always shows up behind our desks while doing so. If only we knew when our CEO was nearby and could get an alert when he was.

Continue reading “Enterprise IoT: Watching Cat Videos Without Getting Caught (or, How I Learned to Stop Looking Over My Shoulder and Trust the CEO Proximity Alert”

Practical Enterprise IoT: The Luminescent Evanescent Apparatus for Voiding Engagements

In this latest installment of our ongoing IoT blog series, we’re going to start discussing a more extended pipeline that we’ve installed here at SnapLogic HQ (as promised in the last post). Space prohibits us from showing all the details of how it works, but we’ll hit the high points in this post and the next.

You may have, at some point in your career, been in a meeting that has gone overtime. (You may be in one right now). You’ve probably wished for something to stop the meeting. Well, the Internet of Things is here to grant your wish.

We’re going to create a pipeline to activate a conference room meeting-is-over signal, the Luminescent Evanescent Apparatus for Voiding Engagements.iot4_graphic1

Continue reading “Practical Enterprise IoT: The Luminescent Evanescent Apparatus for Voiding Engagements”

A Definitionless Definition of Big Data

“‘When I use a word,’ Humpty Dumpty said, in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’

‘The question is,’ said Alice, ‘whether you can make words mean so many different things.'”

Through The Looking Glass, Lewis Caroll

“Big Data”, like most buzzwords, has generated many partially overlapping definitions. (In fact, the author has become of the opinion that just like herds of cows and murders of crows, collections of definitions need their own collective noun. He respectfully submits “opinion”, as in “an opinion of definitions”, as the natural choice.) This post is not about adding another definition of Big Data. It is about considering the operational and architectural implications of calling something Big Data.

http://www.xkcd.com/1429/
Copyright XKCD.

So grab your definition(s) of choice and a representative handful of your data, and consider the following: Continue reading “A Definitionless Definition of Big Data”

Machine Learning for the Enterprise, Part 3: Building the Pipeline

In the last post we went into some detail about anomaly detectors, and showed how some simple models would work. Now we are going to build a pipeline to do streaming anomaly detection.

We are going to use a triggered pipeline for this task. A triggered pipeline is instantiated whenever a request comes in. The instantiation can take a couple of seconds, so it is not recommended for low latency or high-traffic situations. If we’re getting data more frequently than that, or want less latency, we should use an Ultra pipeline. An Ultra pipeline stays running, so the input-to-output latency is significantly less.

For the purpose of this post, we’re going to assume we have an Anomaly-Detector-as-a-Service Snap.  In the next post, we’ll show how to create that Snap using Azure ML. Our pipeline will look like this:

Final Pipeline
Final Pipeline

Continue reading “Machine Learning for the Enterprise, Part 3: Building the Pipeline”

Building an IoT Application in SnapLogic, Part II: Speeding Through the Last Mile

The last post in this ongoing IoT series detailed the creation of a cloud-based Ultra Pipeline to do the bulk of the work for our IoT application. We described the following application:

  • A sensor somewhere (on-premises, from an API, etc.) that produces data that includes a “color” payload;
  • An LED on-premise, attached to our local network, conveniently hooked up to look like a REST endpoint;
  • Two pipelines, one on-premise, one in the cloud.

Continue reading “Building an IoT Application in SnapLogic, Part II: Speeding Through the Last Mile”

Machine Learning in the Enterprise, Part II: Intro to Anomaly Detection

Last time we talked about figuring out what we want machine learning to do to be more important than how to do it. So before we jump into how to build a machine learning pipeline in the SnapLogic Elastic Integration Platform, let’s talk about what we are doing. Continue reading “Machine Learning in the Enterprise, Part II: Intro to Anomaly Detection”

Building an IoT Application in SnapLogic: Figuring out Pipelines and Tasks

In the first post in this series, we talked about the challenges of integrating the Internet of Things into the enterprise. In the next few blog posts, we are going to build a simple IoT application that illustrates all the major aspects of working with SnapLogic and hardware.  In this post, we’re going to skip device details, but at a high level we’ll have:

  • A sensor somewhere (on-premises, from an API, etc.) that produces data that includes a “color” payload;
  • An LED on-premise, attached to our local network, conveniently hooked up to look like a REST endpoint;
  • Two pipelines, one on-premise, one in the cloud.

Hardware Considerations

Some IoT hardware is designed to be cloud-native, and will generally have a publish/subscribe relationship with a cloud server (such as MQTT).  This is very easy to work with from a security standpoint, since the output of these devices are accessible from anywhere.

Other devices instead communicate on their local network.  Assuming your local network isn’t internet accessible, this can create problems in talking to the device.  Usefully, the SnapLogic Control Plane (depicted, in a manner of speaking, as the rightmost rectangle below) comes to our rescue here.

Control-Data Plane Diagram
A “graphical depiction” of the control plane (right) communicating with various data planes, including a Hadooplex at the bottom. We see the artist is somewhat defensive about his rendering of the pachyderm likeness.

Continue reading “Building an IoT Application in SnapLogic: Figuring out Pipelines and Tasks”