Data Mesh Meets Universal Authorization

  • Data discovery mechanism
  • Data quality and trustworthiness
  • Standardization of common infrastructure and reuse of assets
  • Interoperability of domains
  • Self service architecture
  • Observability, governance, and security

Key findings

Data value chain leaders responsible for evaluating and implementing data mesh should:

  • Deploy a collaborative governance platform. Engage all data stakeholders, such that the business, infosec, and data privacy teams work with the data and IT teams to deliver data to business without compromising data security mandates or data privacy regulations.
  • Design a common data access governance layer. Ensure data consumers have consistent access to common data products in different domains through centralized policy management. During the data mesh planning stage, perform proof of concepts of products that provide data governance capabilities. It should not be an afterthought.
  • Implement the universal authorization layer. Permit consumers to search and analyze domain data without performance and scale bottlenecks. The universal authorization layer is typically a best of breed product deployed in a modular and composable architecture, capable of supporting multiple data storage technologies in a hybrid multi-cloud environment.

Data mesh: a brief primer

Several excellent papers have been written on data mesh by its originator, Zhamak Deghani and Thoughtworks. This section only provides a brief high-level overview of its concept and principles.

Data access layer

Interoperability of data is one of the biggest challenges in meeting data mesh’s goal of sharing business context in a self-service manner to maximize its usability. Let’s take an example of a retail organization that has customers’ orders data in the sales domain. A business analyst uses the data to run customer journey analytics models. However, to perform customer churn analytics, the analyst needs to tap into the customer success domain that tracks support tickets, surveys, and social media posts, etc.

  • Share common standards
  • Reuse common resources
  • Reduced integration overhead
  • Develop deep skills in core technologies rather than every department having its own stack.
  • Data discovery
  • Data access governance
  • Data observability

Universal data authorization

Imagine an example where a customer’s name and address are in different domains. This is common in financial services with domains, such as retail banking, wholesale, business banking, lending and leasing, and capital markets. The customer attribute may be called client, account, party, etc. in different domains.

Conclusion

Will data mesh be the panacea for data related issues in the upcoming years? The naysayers are quick to point out that the issues this approach is addressing, and the proposed principles, are not new. While that is true, data mesh brings a fresh perspective. Data quality has been an ever-present issue which the past approaches have failed to alleviate. Data mesh’s domain emphasis provides another approach. It is trite to say that data scientists spend the majority of their time wrangling data. Data mesh’s attempt to treat data as a product can certainly help.

  • Reducing the time between when consumers request new features and when data engineering teams deliver the functionality
  • Fewer ad hoc requests for data on channels such as Slack
  • Higher usage of data when it is made available as a product

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sanjeev Mohan

Sanjeev Mohan

Sanjeev researches the space of data and analytics. Most recently he was a research vice president at Gartner. He is now a principal with SanjMo.