Data mesh is a sociotechnical approach to build a decentralized data architecture by leveraging a domain-oriented, self-serve design (in a software development perspective), and borrows Eric Evans’ theory of

domain-driven design Domain-driven design (DDD) is a major software design approach, focusing on modeling software to match a domain according to input from that domain's experts. Under domain-driven design, the structure and language of software code (class names, ...

and Manuel Pais’ and Matthew Skelton’s theory of team topologies. Data mesh mainly concerns about the data itself, taking the

data lake A data lake is a system or repository of data stored in its natural/raw format, usually object blobs or files. A data lake is usually a single store of data including raw copies of source system data, sensor data, social data etc., and transform ...

and the pipelines as a secondary concern. The main proposition is scaling analytical data by domain-oriented decentralization. With data mesh, the responsibility for analytical data is shifted from the central data team to the domain teams, supported by a data platform team that provides a domain-agnostic data platform.

History

The term ''data mesh'' was first defined by Zhamak Dehghani in 2019 while she was working as a principal consultant at the technology company Thoughtworks. Dehghani introduced the term in 2019 and then provided greater detail on its principles and logical architecture throughout 2020. The process was predicted to be a “big contender” for companies in 2022. Data meshes have been implemented by companies such as

Zalando Zalando SE is a publicly traded German online retailer of shoes, fashion and beauty. The company was founded in 2008 by David Schneider and Robert Gentz and has more than 50 million active users in 25 European markets. Zalando is active in a vari ...

Netflix Netflix, Inc. is an American subscription video on-demand over-the-top streaming service and production company based in Los Gatos, California. Founded in 1997 by Reed Hastings and Marc Randolph in Scotts Valley, California, it offers a ...

Intuit Intuit Inc. is an American business software company that specializes in financial software. The company is headquartered in Mountain View, California, and the CEO is Sasan Goodarzi. Intuit's products include the tax preparation application Tu ...

, VistaPrint,

JPMorgan Chase JPMorgan Chase & Co. is an American multinational investment bank and financial services holding company headquartered in New York City and incorporated in Delaware. As of 2022, JPMorgan Chase is the largest bank in the United States, the w ...

and others.

Principles

Data mesh is based on four core principles: * Domain ownership * Data as a product * Self-serve data platform * Federated computational governance In addition to these principles, Dehghani writes that the data products created by each domain team should be discoverable, addressable, trustworthy, possess self-describing semantics and syntax, be interoperable, secure, and governed by global standards and access controls. In other words, the data should be treated as a product that is ready to use and reliable.{{Cite web , date=2021-12-28 , title=Developments that will define data governance and operational security in 2022 , url=https://www.helpnetsecurity.com/2021/12/28/data-governance-2022/ , access-date=2022-03-01 , website=Help Net Security , language=en-US

References

Databases

History

Principles

See also

References