CASE STUDY
Pepperdata Saves Consumer Internet Company 33% for Apache Spark on Amazon EKS
CASE STUDY
Pepperdata Saves Consumer Internet Company 33% for Apache Spark on Amazon EKS
About the Client
A consumer Internet brand in the Fortune 500 that provides information and services related to financial transactions.
Challenge
The company’s executive leadership team and big data platform team embarked upon an optimization initiative to reduce the rising cost of running Apache Spark application workloads in its massively scaled Amazon EKS environment that was also running Karpenter and YuniKorn.
Solution
Pepperdata Capacity Optimizer’s real-time cost optimization immediately reduced costs by addressing the problem of waste inside the customer’s Spark applications. Capacity Optimizer enabled the Karpenter autoscaler to launch new instances only when existing instances were fully utilized as a result of Pepperdata’s Continuous Intelligent Tuning. Pepperdata also reduced costs by enabling the YuniKorn scheduler to make use of allocated but unused node resources.
Results
In just days, Capacity Optimizer delivered an initial 33 percent reduction of instance hours (vCPU hours) and then continued to deliver this cost reduction on an ongoing basis, equivalent to approximately $100,000 per month in savings.
Executive Summary
The big data platform team at a consumer Internet brand in the Fortune 500 was concerned about ever increasing monthly costs to run Apache Spark application workloads in their Amazon EKS environment that was also running Karpenter and YuniKorn.
The company turned to Pepperdata Capacity Optimizer for autonomous resource optimization. With Pepperdata, they achieved a 33 percent ongoing cost reduction, translating to approximately $100,000 in monthly savings.
Pepperdata’s Continuous Intelligent Tuning enabled the Karpenter autoscaler to launch new instances only when existing instances were fully utilized, and directed the YuniKorn scheduler to use allocated but unused node resources more efficiently.
By automatically delivering greater throughput and measurable cloud cost savings, Pepperdata Capacity Optimizer provided this leading-edge consumer brand with a number of wins, including:
- An immediate and ongoing reduction of costs by 33 percent in a cutting-edge and massively scaled Amazon EKS environment that was also running Karpenter and YuniKorn
- Freedom from manual application tuning, enabling its engineers to focus on higher value-added activities
- Efficiency gains from Pepperdata’s improvement of code for use within the open-source Yunikorn scheduler, with a bonus that Pepperdata contributed knowledge back to the open-source community on its behalf