Over the last few months Gartner has kicked off its Data and Analytics Summit 2022 series, first in London and most recently in Orlando, Florida. One takeaway from these events is that there has been a noticeable increase in messaging from vendors and discussions from Gartner related to Gartner’s data fabric framework.
Unless you’re not member of the data management, analytics, and IT community or have had your head in the proverbial sand if you are, Gartner’s data fabric is largely viewed as a competing decentralization framework to the data mesh decentralization framework founded and advanced by Zhamak Dehghani (now formerly with ThoughtWorks).
Takeaways
There’s nothing unusual with messaging shifts, it’s to be expected and happens all the time. More interesting was the level of confusion that continues to swell in trying to understand the differences between data mesh and data fabric approaches. When I approached this subject with Gartner analysts at Gartner Data and Analytics Summit events, there was an acknowledgement of the amount of hype and discussion surrounding data fabric and data mesh. According to Gartner reps, “they are answering more questions about data mesh versus data fabric than ever before.” At the same time, the assertion was, “data mesh and data fabric can coexist, they’re not mutually exclusive.”
Yet Gartner has a conflicting opinion on data mesh as reflected on its latest Hype Cycle for Data Management (2022), where they marked data mesh as ‘obsolete before plateau’ [sidebar: you’re forgiven if you find this action by Gartner self-serving]. My takeaway from analyst discussions is that Gartner marked data mesh ‘obsolete’ not because they see it going away, but because they see data mesh being absorbed into a data fabric framework given that they claim data fabric is the more comprehensive, and aggregated data governance and master data management approach. In my mind, this claim must be parsed given that Dehghani’s 340+ page book, Data Mesh, Delivering Data-Driven Value at Scale (explaining data mesh) dedicates an entire chapter to Principle of Federated Computational Governance. In the book, governance is indeed accounted for with data mesh, but not in the form of a centralized model.
Nonetheless, with its document, Data Mesh and Data Fabric the Same or Different? (June 2022, G00769853), Gartner attempts to respond to the growing confusion, apparently directed their way from Enterprise architects and data and analytics leaders. Gartner articulates a position (see Figure 1) that both data mesh and data fabric approaches share the same goal of easy access to data, but data fabric delivers, “continuous learning and evaluation using metadata,” while with data mesh, “SME curiosity and completeness determines success or failure.” Gartner further indicates in the document, “as of the first half of 2022, less than 1.5% of the worldwide IT market has direct experience with either approach. Wide variation is expected in terms of practice and delivery over the next 3-5 years.”
So, we are to believe everyone is talking about data mesh and data fabric, but no one is doing it?
It’s all enough to make your head explode.
The most likely reason for the low 1.5% experience level is that, regardless of the framework you wish to follow, data mesh or data fabric, it’s difficult to implement and certainly not an off-the-shelf product that can be bought, as most supporters (and distractors) of data mesh understand.
Federated Integration: The Key to Achieving a Data Mesh or Data Fabric
Another observation from the recent Gartner Data and Analytics Summit in Orlando, there was relatively more positioning of technology targeted at both IT and business groups, as opposed to just IT. It’s a reasonable assumption that data mesh buzz had something to do with this as well. Business-group domain ownership of data is one of the four main principles of the data mesh concept (the other three principles being: data as a product, self-service, and federated computational governance).
These principles are easy to say, but hard to deliver entirely via a single product.
Hence, at SnapLogic we’ve embraced a federated model through designing experiences that are specific for IT, business users, and data professionals, while enabling logical separation of domains when desired. For example, with each SnapLogic platform instance, you can create workspaces, which we call Orgs. Each Org has logical separation: business teams can have their own Org, IT can have an Org, etc. Then all Orgs, although distributed, can be globally monitored and managed.
We employ a visual, graphical design approach to declarative SQL. We augment this with AI assisted integration that performs next-step recommendations when creating a data flow or business process pipeline. Together, these capabilities dramatically simplify integrations for all. Even our most experienced users indicate they save as much as 50 to 60% of their time when constructing pipelines using our Iris AI-powered integration assistant as compared to not using it.
SnapLogic users save even more time when compared to manual coding. This makes self-service at scale possible. Our largest installations have hundreds, if not thousands, of individuals performing their own integrations.
We also combine data integration, app-to-app integration, and API development/management (APIM) in one platform.
Want complete separation of teams and go full decentralization? Want to share data products?
APIs will be key to achieving this and offering APIM as part of the SnapLogic integration experience will make your life easier.
We prefer to look through the confusion and not debate one approach versus the other, but rather bring you one step closer to a data mesh and data fabric framework – if this is the direction you’re headed. As we see it, you want business agility and acceleration, at scale, but you can do without the crazy complexity that comes with the modern data platform approaches of today.
In the words of our visionary CEO, Gauarv Dhillon, “modern data platforms are built, not bought.”
Experience the leading self-service integration platform for yourself by starting your 30-day free trial.
