The Evolution of IOT Data Management with Joseph A di Paolantonio

Today’s post comes from Joseph A. di Paolantonio, an industry expert working at the convergence of IoT with data management and analytics at DataArchon.com and the Boulder BI Brain Trust. Leveraging a career that started with renewable energy research in graduate school and industry, developing risk assessment models and algorithms for aerospace systems, and managing teams for enterprise data warehousing, BI and data science, Joseph is defining sensor analytics ecosystems to bring value from the IoT.

What’s IOT All About

We have been asking what the IoT is all about for a very long time. Since Kevin Ashton first coined the phrase in 1999, and perhaps even since Nikola Tesla first played with a remote control boat in 1898. For many, the simple act of connecting a device, not a computer, not a router, to the Internet is enough. But even if everything around us, in work, at home, for business, for play, was connected..Does this make the Internet of Things? Does this fulfill much of the hype or any of the stories surrounding the IoT? Not at all.

Continue reading “The Evolution of IOT Data Management with Joseph A di Paolantonio”

Streaming Data and Data Lakes at #StrataHadoop World

Ravi DSnapLogic’s big data expert and Head of Enterprise Architecture, Ravi Dharnikota, was featured on Information Management recounting his observations at last month’s Strata+Hadoop World in San Jose. The main takeaway was that the attendees and sessions were primarily focused on streaming data, data lakes, and Apache Spark for analytics. He noted: “While the continuous innovation and change in the big data industry provides fast, frequent improvements to the technology, it is tough to keep up with in an organization where there are competing priorities and projects.”

You can read the full Q&A below.  Continue reading “Streaming Data and Data Lakes at #StrataHadoop World”

Will the Data Lake Drown the Data Warehouse?

Mark MadsenThis is the first post in a series that will feature extracts from the new whitepaper: Will the Data Lake Drown the Data Warehouse? The paper is written by Mark Madsen, founder and president of Third Nature. Third Nature is a consulting and advisory company specializing in analytics and information management and the technology infrastructure required to support them. Mark Madsen is a well-known consultant and industry analyst who frequently speaks at conferences and seminars in the US and Europe and writes for a number of leading industry publications. Continue reading “Will the Data Lake Drown the Data Warehouse?”

The Impact of SMACT on the Data Management Stack

I had the opportunity to present to the San Francisco Bay Area Chapter of the Data Management Association (DAMA) this week on the topic of the changes to today’s data management stack. We discussed why CIOs and IT organizations in general are getting SMACT and reviewed some of the new integration challenges of the enterprise:

  • Big data access and analytics,
  • Disconnected SaaS silos,
  • API proliferation,
  • and the brewing storm of data represented by the Internet of Things.

benioff_speed_df14We then reviewed some of the characteristics and challenges posed by what we call the Integrator’s Dilemma and why old approaches to data management and same old same old (SO, SO) approaches to cloud and big data integration are not going to cut in the modern enterprise. What’s different, you ask? Here are 5 changes we discuss with our customers and partners at SnapLogic:

  1. Speed: As Marc Benioff recently noted at Dreamforce 2014, “Companies are no longer competing against each other. They’re competing against speed.”  Speed was also identified as the #1 reason companies choose an integration platform as a service (iPaaS), according to our recent TechValidate survey.
  2. User (and Buyer) Expectations: Self-service is a hot topic in the world of analytics, big data integration and iPaaS. We’ve written about the rise of the citizen integrator regularly on this blog. Gartner has recently published a report on the topic: Embrace the Citizen Integrator Approach to Improve Business Users’ Productivity and Agility.
  3. The Data: Of all of the changes in information management infrastructure, what’s new when it comes to big data volume, variety and velocity has been the most widely covered. See the original post on this topic from Gartner’s Doug Laney here.
  4. Cloudification and Data Gravity: With 2015 technology predictions season about to kick in, with so much recent attention on the shift to cloud analytics, here’s what Forrester has to say about the state of cloud adoption:  ”In 2015, cloud adoption will accelerate and technology management groups must adapt to this reality by learning how to add value to their company’s use of these services through facilitation, adaptation and evangelism. The days of fighting the cloud are over. This means major changes are ahead for you, your application architecture, portfolio, and your vendor relationships.”
  5. Standard, Protocols and Architectural Styles: We’ve talked about why the enterprise service bus (ESB) doesn’t fly in the cloud and written extensively about JSON and REST.  We’ve also written about why an ELA with a legacy data management vendor in 2014 is like betting on COBOL in 1998. As Craig Stewart, our Sr. Director of product management likes say: “It’s easy to put rows and columns into a document, but vice-versa doesn’t work.”

So what’s the same? Successful cloud application and big data platform adoption requires the right plumbing. Whether it’s orchestrating and streaming data between applications in real time or acquiring, preparing and delivering big data to downstream analytics tools and users, it’s still all about the pipes. While many of the circumstances and pace have changed, the burden of ensuring your data is timely, relevant and trustworthy has not. Your integration solution must be an innovation on ramp, not a roadblock.

I’ve embedded the presentation below, which includes a few recommendations for data management practitioners. Feedback appreciated.