BENCHMARKS
Pepperdata Reduces the Cost of Running Spark Workloads on Amazon EKS at Scale by 41.8%
BENCHMARKS
Pepperdata Reduces the Cost of Running Spark Workloads on Amazon EKS at Scale by 41.8%
Using the TPC-DS benchmarking framework, Pepperdata set up a Spark testing environment on Amazon EKS to test the performance and effectiveness of Pepperdata Capacity Optimizer for the following metrics:
- Total instance hours consumed
- Total workload run time
- Average run time per application
- Average concurrent container count
Key Findings
-
Total Instance Hours Consumed
Decreased by 41.8% -
Total Workload Run Time (minutes)
Reduced by 45.5% -
Average Run Time per Application
Decreased by 22.2% -
Average Concurrent Container Count
Increased by 26.2%
Takeaways from the TPC-DS Benchmarking Results
Capacity Optimizer enabled more applications to run in a given time period, which increased the overall throughput of the environment. After enabling Capacity Optimizer, the entire TPC-DS workload:
- Ran 45.5 percent faster (as measured by decreased run time)
- Cost 41.8 percent less (as measured by reduced instance hour consumption)
For more insight into how Pepperdata Capacity Optimizer enhanced the performance and efficiency of Apache Spark workloads on Amazon EKS at scale, download the report.