Data Mesh In Action

Data Mesh is a modern data management paradigm centered around product thinking, decentralized data ownership, decentralized architecture, and federated governance. These concepts aim to help organizations deliver data products at scale with high autonomy without compromising compliance and governance. While the value of the Data Mesh paradigm is widely recognized, materializing it is difficult due to the established data ecosystems and heavily centralized architecture. 

If you already have established data architecture and systems, the best way to implement Data Mesh in your organization is by following a top-down approach where the initial focus is the experience of data producers and consumers. As the process continues, more fundamental changes occur in lower stacks. At the high level, this approach can be summarized as follows:

• Define the self-service experience for data producers and consumers.

• Embed business requirements such as federation, compliance, and RBAC.

• Find the integration points in the existing system to collect, augment, and surface metadata, key indicators for the data product, that is important for your business and decision-making process. The key here is low-touch to ensure business continuity and smoother and faster adoption. 

• Revisit lower components of the system such as pipelines, jobs, data ingestion mechanisms, orchestration, and other tooling to converge it as much as possible to what was defined as a data product, a self-contained unit.

• Make this process continuous and iterative to maintain alignment with the business.

As an execution strategy, while making changes, it is best to start small, prove value, and add more. There needs to be a balance between re-using, re-purposing existing tooling and systems or creating new ones. We recommend investing in the experience layer and leveraging solutions purpose-built for surfacing Data Mesh concepts natively.