Autonomously optimize your Apache Spark cluster resources on top of Managed Autoscaling, Karpenter, Spark Dynamic Allocations, and other traditional optimization efforts such as manual tuning. Pepperdata delivers:
Autonomously optimize your Apache Spark cluster resources on top of Managed Autoscaling, Karpenter, Spark Dynamic Allocations, and other traditional optimization efforts such as manual tuning. Pepperdata delivers:
Autonomously optimize your Apache Spark cluster resources on top of Managed Autoscaling, Karpenter, Spark Dynamic Allocations, and other traditional optimization efforts such as manual tuning. Pepperdata delivers:
If you’re running Spark, give us 6 hours, we’ll save you 30% or more on top of everything you’ve already done.
Running Pepperdata Capacity Optimizer alongside the autoscaler for Amazon EMR on Amazon EKS helped reduce a customer’s instance hours by 42.5%. Here are the key findings with Pepperdata:
Saved in one month
Reduced instance hours
Improved normalized core efficiency
After Karpenter deploys the right instance for your workload, Pepperdata saves a further 30% in costs by preventing Spark applications from wasting requested resources at the task level.
Here’s how Capacity Optimizer does it:
Pepperdata provides the cluster scheduler with real-time data on available capacity
Your scheduler adds more jobs to existing instances without spinning up new instances
When new apps come along, new instances are spun up only when existing ones are truly full
Cost Savings: Reduced instance hour consumption
Improved Performance: Decreased application runtime
Increased Throughput: Uplift in average concurrent container count
*TPC-DS is the Decision Support framework from the Transaction Processing Performance Council. TPC-DS is an industry-standard big data analytics benchmark. Pepperdata’s work is not an official audited benchmark as defined by TPC. TPC-DS benchmark results (Amazon EKS), 1 TB dataset, 500 nodes,
10 parallel applications with 275 executors per application.
Looking for a safe, proven method to reduce waste and cost by up to 47% and maximize value for your cloud environment? Sign up now for a free cost optimization demo to learn how Pepperdata Capacity Optimizer can help you start saving immediately.