Machine Learning for the Enterprise, Part 3: Building the Pipeline

In the last post we went into some detail about anomaly detectors, and showed how some simple models would work. Now we are going to build a pipeline to do streaming anomaly detection.

We are going to use a triggered pipeline for this task. A triggered pipeline is instantiated whenever a request comes in. The instantiation can take a couple of seconds, so it is not recommended for low latency or high-traffic situations. If we’re getting data more frequently than that, or want less latency, we should use an Ultra pipeline. An Ultra pipeline stays running, so the input-to-output latency is significantly less.

For the purpose of this post, we’re going to assume we have an Anomaly-Detector-as-a-Service Snap.  In the next post, we’ll show how to create that Snap using Azure ML. Our pipeline will look like this:

Final Pipeline
Final Pipeline

Continue reading “Machine Learning for the Enterprise, Part 3: Building the Pipeline”

Machine Learning in the Enterprise, Part II: Intro to Anomaly Detection

Last time we talked about figuring out what we want machine learning to do to be more important than how to do it. So before we jump into how to build a machine learning pipeline in the SnapLogic Elastic Integration Platform, let’s talk about what we are doing. Continue reading “Machine Learning in the Enterprise, Part II: Intro to Anomaly Detection”

Enterprise IoT: Defensive Development

In the initial post in this series, we talked about how IoT data, once acquired, is still just data. Before diving into building our first demo, it’s worth taking a moment to talk about getting the data to the stage of “once acquired”.

If you’re an existing SnapLogic customer – and if, not, why not request a trial? – you’re used to working in an environment where our Snaps work hard to ensure you aren’t confronted by irrational application behavior, and things Just Work. This happens because our talented developers get to spend hundreds or thousands of person-hours on creating integrations for a wide variety applications, APIs and enterprise data sources.

XKCD.com - Good Code Flowchart
Courtesy xkcd.com

For IoT devices, though, we provide thoroughly-tested Snaps to enable communication with the device, but we can’t guarantee the device itself won’t do silly things. One such silly thing occurred during the creation of the demo this blog series is going to show you how to build.  An addressable color-changing USB LED was used that can theoretically display any of over 16 million colors. The included demo software lets you pick any of those colors and see the light change. However, when we integrated it with SnapLogic, we got odd errors back when asking it to change color.
Continue reading “Enterprise IoT: Defensive Development”

Practical Machine Learning for the Enterprise, Part I

“Speak English!” said the Eaglet. “I don’t know the meaning of half those long words, and, what’s more, I don’t believe you do either!” – Alice in Wonderland

Machine learning (and its subset, deep learning) have been hailed as The Next Big Thing, capable of creating autonomous cars, upending business models, and generally requiring a massive investment in human and financial capital for a business to stay competitive. The hype has drowned out the ‘how’ and particularly the ‘why’. While we are bullish at SnapLogic on the promise of machine learning (ML) in the enterprise, we think the first question is not “how do I implement it?” but “what is it I want to know?”

A Crash Course in ML

At its core, most ML algorithms are based on something you may have done in high school: drawing a line through a bunch of points. In fact, if you’ve ever run a regression in Excel, you’ve done machine learning. So what’s the big deal now? Continue reading “Practical Machine Learning for the Enterprise, Part I”