5 Data Predictions for 2016

We all know that huge amounts of data are being turned into information, and then insight, allowing businesses to redefine industries and win over today’s consumers. And yet most companies are still dabbling on the edges, still fighting against restrictive silos and old IT. As you evaluate your data strategy for 2016, here are four data predictions to keep in mind, as well as a look on the start-up industry feeding us data solutions.

modern_integration1) Data Eats the World and Integration Strategies Will Drive Digital Transformation
With the rise of companies like Uber, Netflix, and services like Amazon Prime, data is truly eating the world. A strong integration strategy and vendor-agnostic solutions will help enterprises emulate these disruptors, help them overcome their data challenges and flourish.

Digital transformation needs to be powered by the right integration and orchestration services. It will fail unless integration is brought to the surface. Whether it’s your big data initiative, cloud application strategy, IoT initiative, etc., integration can be the bottleneck or the on-ramp. Old pipes must be dealt with – ignore them at your own peril.

Integration itself must be self-service. It has to come out of the basement into the front office. It can’t just be isolated groups of developers slinging code; it needs to be many more people from lines of business who have secure, self serve data access.

2) Rising Data Lakes will Drown the Warehouse
2016 will be the year of the data lake. It will surround and, in many cases, drown the data warehouse. And we’ll see significant technology innovations, methodologies and reference architectures that turn the promise of broader data access and big data insights into a reality. At the same time, big data platforms will mature (read: security, governance, metadata management) and go beyond being primarily developer tools for highly skilled programmers. The data lake will allow organizations to track and manage data they’ve never had access to in the past. New data management strategies are already leading to more predictive and prescriptive analytics that are driving improved customer service experiences, cost savings and an overall competitive advantage when there is the right alignment with key business initiatives.

In 2016, data streaming will become an even bigger deal for enterprises, as they will need to feed data from new systems and things, and from their traditional warehouse into their data lake in order to provide additional, real-time supply or risk the business losses that come with not conducting analytics in real-time. For the delivery and consumption the new technology stack (wrangling, ingesting, even ETL/ELT), transformations made in the data lake (Hadoop, Spark, etc.) will enable this to be done much cheaper and faster. Thus far, stream processing has been dominated by open source technologies such as Kafka, Storm and Spark Streaming. The leading big data processing vendors will almost certainly respond and try to make stream processing first class functionality within their platforms. Modern integration platforms will also need to be able to handle streaming data requirements.

snaplogic_data_lakeSink in a data warehouse or swim in a data lake? The Enterprise Data Warehouse (EDW) will be in maintenance mode – and in some cases life support. It will continue to do what it’s good at: operational and historical reporting and analysis (aka rear view mirror) especially for financial reporting, but will start to be eclipsed by data lake strategies in sales, marketing and other business functions. More dynamic, poly-structured data will move to the lake – and that environment will become the center of data gravity in most enterprises.

3) IoT Goes from Over-Hyped to Emerging Reality
Internet of Things (IoT) is the most disruptive trend in data management, not just because it’s at the peak of its hype cycle, but because of the sheer volumes and new types of data that organizations need in order to access, store, administer, monitor and measure IoT data in real-time as part of their new analytics infrastructure.

The IoT era is upon us, but in 2016, we’ll be connected in even more ways than we’ve ever imagined. It will be commonplace for our wearables to tell our smart thermostats how to adjust according to our homeostasis. Our cars will be able to predict health concerns like oncoming seizures or high blood alcohol levels. And with all this data, companies will rely on a system that ties this entire ecosystem together and make sense of all the information coming from all these.

The debate about “appropriate” use of personal information and data security will continue but the staggering benefits of using these data in aggregate will outweigh some of these considerations as we continue to wire up sensors, machines and people across the planet.

4) Multi-Cloud is the New Reality
Microsoft Azure, Google and Amazon provide enterprises the ability to deploy multi-cloud solutions. It’s no longer your cloud vs. my premises, as companies deploying multiple SaaS and PaaS technologies in a heterogeneous architecture is becoming common now. Multi-cloud strategies will be a regular part of 70% of all companies’ data infrastructure by 2019, as Gartner predicts. Enterprises need a multi-cloud strategy and see neutral service and technology providers, a Switzerland, if you will, as a necessity to help unify data from each cloud-based program. Legacy vendors will need to adopt the same plan if they hope to remain relevant and useful to enterprises moving forward.

5) Many so-called Unicorns (private companies valued at over $1Billion) Will Turn Into Unicorpses
There’s been so much hype around the promise of new technologies and disruptive business models in 2015 that the unicorns of the industry are going to become unicorpses if they can’t demonstrate a fast time to value, a quantifiable ROI and a path to a sound economic model.

The industry will increasingly realize that need to get out of the casino business and back into the technology business. If we’ve learned anything from the lackluster tech IPOs of 2015, it’s the return of substance.

There you have it. My fearless forecast. I wish you all the best in 2016. Happy New Year!

  • Good stuff Gaurav…agree with the “unicorpse” prediction!