The Challenge

BMW, a German luxury automobile manufacturer, aimed to localize their approach to aftermarket sales of spare parts. Initially launched as a European initiative, local sales regions were granted access to global order management systems to plan and track spare parts sales. While global systems are excellent at standardizing business processes, they often fall short in helping local managers interpret data within a regional context.

To address this, it became necessary to integrate global process data with regional insights.

The Approach

Data Governance

Our team, assigned to one of the European regions, focused on accessing the global data lake through a structured data access and governance platform. This platform enabled us to retrieve order data with minimal delays. Since the platform is built on AWS technology, we utilized the AWS Glue Data Catalog as a metadata catalog for the data lake.

Localizing global data

Operating under strict governance controls, we used a spoke AWS account to store and process local data from several unique source systems, integrating it with global order data. We further leveraged the AWS ecosystem, using AWS Glue for data transformation and replication. AWS Glue Jobs orchestrated the 50+ pipelines involved, ensuring reliable data storage on AWS S3.

All data transformation code was written in Python, with lifecycle management handled via Terraform.

This approach resulted in a local data lake, built and governed using the same principles as the global one, but housed within a spoke account.

Fast reporting

Given that data lakes are optimized for quick data storage but often struggle with slow set-based operations, we introduced an additional component to accelerate reporting.

We employed Amazon Athena, a SQL engine based on Apache Presto, to query the data lake files. The chosen business intelligence tool was integrated as a client application, allowing business analysts to access localized insights with minimal latency.

The architecture

The Results

After a 16-week project, analysts had local insights at their disposal, including a solid and auditable change management system to track changes in definitions and transformations of data.