BENCHMARKS

Pepperdata Reduces the Cost of Running Spark Workloads on Amazon EKS at Scale by 41.8%

eks benchmarking 1

BENCHMARKS

Pepperdata Reduces the Cost of Running Spark Workloads on Amazon EKS at Scale by 41.8%

Using the TPC-DS benchmarking framework, Pepperdata set up a Spark testing environment on Amazon EKS to test the performance and effectiveness of Pepperdata Capacity Optimizer for the following metrics:

  • Total instance hours consumed
  • Total workload run time
  • Average run time per application
  • Average concurrent container count

Key Findings

Screen Shot 2023 10 17 at 12.46 1 1
  • Group 2726

    Total Instance Hours Consumed
    Decreased by 41.8%

  • Group 2764

    Total Workload Run Time (minutes)
    Reduced by 45.5%

  • Group 2780

    Average Run Time per Application
    Decreased by 22.2%

  • Group 2784

    Average Concurrent Container Count
    Increased by 26.2%

Takeaways from the TPC-DS Benchmarking Results

Capacity Optimizer enabled more applications to run in a given time period, which increased the overall throughput of the environment. After enabling Capacity Optimizer, the entire TPC-DS workload:

  • Ran 45.5 percent faster (as measured by decreased run time) 
  • Cost 41.8 percent less (as measured by reduced instance hour consumption)

For more insight into how Pepperdata Capacity Optimizer enhanced the performance and efficiency of Apache Spark workloads on Amazon EKS at scale, download the report. 

 

Explore More

Looking for a safe, proven method to reduce waste and cost by up to 47% and maximize value for your cloud environment? Sign up now for a free cost optimization demo to learn how Pepperdata Capacity Optimizer can help you start saving immediately.