Challenge
Controlling the month-over-month spend increase for Snowflake during increased business turbulence.
Our client has their data platform based on Snowflake; a data platform built offered as a service. They value the platform’s ability to start small and scale performance and spend while increasing the workload on the platform.
In times of surging business demand due to the COVID-19 outbreak in 2021, the data platform team refocused on delivering new business features while spreading architectural focus across initiatives. The client could not benefit from consolidating workloads in one common platform, resulting in a sudden month-over-month cost increase of 25%.
Thanks to our initial platform setup, several warning signs were triggered early on, and the client decided to engage us for a cost optimization assignment.
Approach
Deploy our in-house framework for spend visibility and adapt the architecture to benefit from performance optimizations.
Tropos has a battle-tested blueprint architecture for Snowflake deployments. We started multiple Snowflake projects from scratch and have adapted many along the way.
We used a specific approach to address the assignment:
- Analyze the as-is situation using our internal tools;
- Defining the desired target state for the client, i.e., controlling spend and understanding spend drivers better;
- Define and perform actions based on the gap analysis between both;
The assignment involved desk research, a deep understanding of Snowflake’s data ingest, data processing, and integration features. It was entirely led by one senior architect with development capacity provided by one senior consultant.
Results
A transparent spend model backed by an optimized architecture and periodical review.
After an assignment of 2 days, technical optimizations worth two days, and a monitoring period, the client could post considerable spending savings.
The architecture was optimized based on changing workloads.
As the business demands and technology tool stack had rapidly changed throughout the project, we decided to make material adjustments in the way data was processed in the platform. Rather than anticipating questions and precalculating answers, we decided to focus even more on data model consistency and calculate answers at the request time.
We delivered a transparent paradigm for project spending and charge through.
Thanks to the optimized architecture, the client can now bill internal and external clients based on their actual platform usage.
We brought in monthly spend monitoring and advisory services.
The data platform is in full expansion. As the client wants to maximize Snowflake’s value for their budget, we delivered a standardized monthly spend check-in which was reviewed.
- Cost deviations from the plan;
- The impact of new architectural decisions on the overall deployment;
- Spend saving opportunities because of new Snowflake feature releases;
