Learn how we helped our client set up a data mesh architecture
Our client was a membership-based organization offering a unique combination of advocacy and service-oriented mobility, travel, and leisure activities. It supports its members with services such as organized travel and holiday guides, roadside assistance, insurance, driving courses, and traffic information. Committed to sustainable societal development, the organization connects with its members through a network of over 80 stores, its website, and a dedicated Member Service Center.
The centralized data warehouse team, responsible for handling all the reporting inquiries for its subsidiaries, was struggling to keep up with business demand as the complexity of data requests increased. As a result, more than 20 subsidiaries developed their own internal data analytics capabilities, each going through independent learning curves and investing in separate software and infrastructure. This fragmentation led to missed collaborative learning opportunities and unnecessary costs due to missing out on economies of scale. On the other hand, varying regulatory frameworks called for a decentralized model.
Combining these demands, the organization decided to adopt a Data Mesh approach. A Data Mesh is a decentralized data architecture that treats data as a product and emphasizes domain-oriented ownership, self-service data infrastructure, and federated governance. This approach shifts data management from a centralized data team to domain-specific teams, enabling scalability and agility in large organizations. It balances decentralization with organization-wide standards and policies, ensuring interoperability, security, and compliance across all data domains.
Snowflake was procured as the future data platform consolidating all subsidiaries’ needs. The internal Snowflake platform owner chose to partner with Tropos as a strategic ally to facilitate the design and deployment of their Snowflake platform. This collaboration aims to establish strong governance practices and implement logical data mesh operating processes from the outset.
To help support the incremental development of the new data platform, we helped define key platform capabilities and introduced our best practices and, operational procedures.
For this assignment, we collaborated mainly with the technical architect and lead platform architect within the central data team of the client.
First, we conducted a full audit of the existing Snowflake deployments. Then, we developed a roadmap for designing a data platform that maximizes the independence of the client's subsidiaries while ensuring that the platform owner can apply and update centrally governed controls.
Innovations:
With a range of data transformation technologies scattered all across the organization, the customer was looking for opportunities to harmonize.
We have co-created an operating model that allows maximum independence for every individual subsidiary to organize analytics the way they want, and meanwhile making sure the central team has control over the way data is shared over an internal data marketplace, a data sharing mechanism that implies roles and responsibilities for both producer and consumer. The practice is based on our in-house developed blueprint for internal data marketplaces.
The deliverables included guidance on code-first workflows, data contracts, and principles on centralizing data processing whilst keeping regulatory guardrails in place.
Are you looking for a strategic data services partner with expertise in designing and deploying a scalable, compliance-ready data platform for your organization? A partner experienced in establishing strong governance practices from the outset for organizations that want to transition to a data mesh approach? A partner that equips your internal data team with best practices and a blueprint for robust governance even after our engagement concludes?
Don't hesitate to contact us—let's discuss how we can support you!
.avif)