Data as a Product: How Data Mesh is Decentralizing Data Architectures
Keywords:
Data ownership, data silosAbstract
Data Mesh represents a transformative shift in data architecture, addressing the limitations of centralized data management by embracing decentralization and treating data as a product. Traditional data systems, such as centralized data warehouses and lakes, often need help with scalability, agility, and meeting the diverse needs of modern organizations. These centralized approaches can create bottlenecks, delay insights, and limit the ability to respond to changing business demands. Data Mesh reimagines this paradigm by decentralizing data ownership and aligning it with specific business domains. This domain-oriented approach ensures that those who best understand the data are responsible for its quality, usability, and maintenance, promoting a culture of accountability & ownership. By applying principles of product thinking, data in a Data Mesh architecture is developed and managed with end-users in mind, ensuring it meets their needs consistently and reliably. A key enabler of this model is a self-serve data infrastructure, providing teams with the tools, platforms, and frameworks needed to independently create, maintain, and share data products without reliance on centralized teams. This decentralization is governed by federated principles, balancing autonomy and adherence to enterprise-wide standards, ensuring data consistency, security, and compliance. The result is a scalable, agile, and democratized data ecosystem that empowers organizations to extract actionable insights more efficiently and effectively. However, adopting a Data Mesh framework is challenging, including cultural shifts, the need for specialized skills, and managing the technical complexity of distributed systems. Despite these hurdles, Data Mesh provides a robust foundation for organizations to meet the demands of a rapidly evolving data landscape, enabling them to innovate faster and make more informed decisions. By decentralizing ownership & embracing a product-oriented mindset, Data Mesh unlocks the full potential of data, paving the way for organizations to thrive in a data-driven era while fostering greater collaboration and innovation across domains.
Downloads
References
Wang, C., & Jiang, P. (2015, June). The approach of hybrid data on tag in decentralized control system. In 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER) (pp. 799-802). IEEE.
Burdeniuk, A. (2014). A mesh architecture for data management of matrix computations (Doctoral dissertation).
De Filippi, P. (2015). Community mesh networks: citizens' participation in the deployment of smart cities. In Handbook of research on social, economic, and environmental sustainability in the development of smart cities (pp. 298-314). IGI Global.
Raoult, B., Aubert, G., Gutiérrez, M., Arciniegas-Lopez, C., & Correa, R. (2009). Virtual organisation in the SIMDAT meteorological activity: a decentralised access control mechanism for distributed data. Earth science informatics, 2, 63-74.
Trieb, R., Ballester, A., Kartsounis, G., Alemany, S., Uriel, J., Hansen, G., ... & Vangenabith, M. (2013, November). EUROFIT—integration, homogenisation and extension of the scope of large 3D anthropometric data pools for product development. In 4th International conference and exhibition on 3D body scanning technologies, Long Beach, CA, USA (pp. 19-20).
Raval, S. (2016). Decentralized applications: harnessing Bitcoin's blockchain technology. " O'Reilly Media, Inc.".
Wheeler, A., & Auburn, C. B. (2003). Wireless mesh networks: better connectivity: to ensure reliability in industrial applications, a wireless data-transmission network must be'self-organizing'and'self-healing.'(eBusiness for the Chemical Process Industries). Chemical Engineering, 110(5), 73-78.
Khatri, V. (2016). Managerial work in the realm of the digital universe: The role of the data triad. Business Horizons, 59(6), 673-688.
Nagel, B., Böhnke, D., Gollnick, V., Schmollgruber, P., Rizzi, A., La Rocca, G., & Alonso, J. J. (2012, September). Communication in aircraft design: Can we establish a common language. In 28th International Congress of the Aeronautical Sciences (Vol. 201, No. 2).
Duncan, R. (1990). A survey of parallel computer architectures. Computer, 23(2), 5-16.
Ahlbrandt, J., Brammen, D., Majeed, R. W., Lefering, R., Semler, S. C., Thun, S., ... & Röhrig, R. (2014). Balancing the need for big data and patient data privacy–an IT infrastructure for a decentralized emergency care research database. In e-Health–For Continuity of Care (pp. 750-754). IOS Press.
Stolpe, M. (2016). The internet of things: Opportunities and challenges for distributed data analysis. Acm Sigkdd Explorations Newsletter, 18(1), 15-34.
Lyko, K., Nitzschke, M., & Ngonga Ngomo, A. C. (2016). Big data acquisition. New horizons for a data-driven economy: a roadmap for usage and exploitation of big data in Europe, 39-61.
Shang, W., Bannis, A., Liang, T., Wang, Z., Yu, Y., Afanasyev, A., ... & Zhang, L. (2016, April). Named data networking of things. In 2016 IEEE first international conference on internet-of-things design and implementation (IoTDI) (pp. 117-128). IEEE.
Bonifati, A., Chrysanthis, P. K., Ouksel, A. M., & Sattler, K. U. (2008). Distributed databases and peer-to-peer databases: past and present. ACM SIGMOD Record, 37(1), 5-11.
Gade, K. R. (2019). Data Migration Strategies for Large-Scale Projects in the Cloud for Fintech. Innovative Computer Sciences Journal, 5(1).
Gade, K. R. (2018). Real-Time Analytics: Challenges and Opportunities. Innovative Computer Sciences Journal, 4(1).
Komandla, V. Enhancing Security and Fraud Prevention in Fintech: Comprehensive Strategies for Secure Online Account Opening.
Komandla, V. Transforming Financial Interactions: Best Practices for Mobile Banking App Design and Functionality to Boost User Engagement and Satisfaction.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
License Terms
Ownership and Licensing:
Authors of research papers submitted to Distributed Learning and Broad Applications in Scientific Research retain the copyright of their work while granting the journal certain rights. Authors maintain ownership of the copyright and have granted the journal a right of first publication. Simultaneously, authors agree to license their research papers under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License.
License Permissions:
Under the CC BY-NC-SA 4.0 License, others are permitted to share and adapt the work, as long as proper attribution is given to the authors and acknowledgement is made of the initial publication in the journal. This license allows for the broad dissemination and utilization of research papers.
Additional Distribution Arrangements:
Authors are free to enter into separate contractual arrangements for the non-exclusive distribution of the journal's published version of the work. This may include posting the work to institutional repositories, publishing it in journals or books, or other forms of dissemination. In such cases, authors are requested to acknowledge the initial publication of the work in this journal.
Online Posting:
Authors are encouraged to share their work online, including in institutional repositories, disciplinary repositories, or on their personal websites. This permission applies both prior to and during the submission process to the journal. Online sharing enhances the visibility and accessibility of the research papers.
Responsibility and Liability:
Authors are responsible for ensuring that their research papers do not infringe upon the copyright, privacy, or other rights of any third party. Scientific Research Canada disclaims any liability or responsibility for any copyright infringement or violation of third-party rights in the research papers.
If you have any questions or concerns regarding these license terms, please contact us at editor@dlabi.org.
