Improve Cluster Autoscaler Efficiency with Capacity Optimizer

Pepperdata Capacity Optimizer automatically optimizes your cluster resources, recapturing wasted capacity so you can run more applications and get the most out of your infrastructure investment. Capacity Optimizer enables you to:

  • Get up to 3X price-performance improvement on top of AWS autoscaling.
  • Recapture wasted capacity, run more applications, and get the best ROI from your infrastructure investment.
  • Optimize each node’s ability to run an optimal number of containers and to run the same number of workloads on fewer instances.

Immediately Improve Big Data Cluster Throughput

On a typical cluster, Capacity Optimizer uses machine learning (ML) to make thousands of decisions per second, analyzing the resource usage of each node in real time. The result: CPU, memory, and I/O resources are automatically optimized to increase utilization, and waste is eliminated in both Kubernetes and traditional big data environments. Capacity Optimizer rapidly identifies where more work can be done and adds tasks to nodes with available resources. Even the most experienced operator dedicated to resource management can’t make manual configuration changes with that precision and speed.

photo content image 7

Boost Autoscaling Performance and Reduce Your Cloud Costs

In cloud environments, autoscaling provides the elasticity you need for your big data workloads, but it often leads to uncontrolled costs. Cloud providers provision infrastructure based on the peak needs of workloads. This guarantees that maximums are met but can create a lot of provisioning waste—the very waste that Capacity Optimizer identifies and returns to you in the form of optimized, available resources to run more jobs. 

Whatever your cloud platform, Capacity Optimizer uses autonomous optimization to intelligently augment autoscaling and ensure that all nodes are fully utilized before additional nodes are created. The net effect is that horizontal scaling is optimized and waste is eliminated.

Auto Scaling Benchmark Report

Our latest benchmark results using HiBench workloads with Pepperdata Capacity Optimizer on Amazon EMR show that you can dramatically lower your costs and improve CPU and memory utilization.

Automatically Reduce Your Amazon EMR Costs

Capacity Optimizer complements traditional EMR autoscaling by reducing resource waste on your cluster before EMR autoscaling is enabled. On top of Amazon EMR, Capacity Optimizer can reduce the number of cores by up to 63%, active nodes by up to 67%, and CPU idle time by up to 30%. Capacity Optimizer is part of the Pepperdata product suite available for free on AWS Marketplace. Pepperdata for EMR allows you to:

  • Automatically tune your cloud deployment for optimal performance.
  • Get full-stack observability, automated tuning, and job-specific recommendations for Spark and MapReduce.
  • Automatically optimize node performance, and prevent waste by applications.
  • Customize alerts to quickly understand and troubleshoot application and infrastructure issues.

Pepperdata Helps Fortune 100 Financial Services Giant Gain Control Over Their Runaway Data Infrastructure Spend

A multinational investment bank was growing at an exponential rate and faced exploding costs. Read how Pepperdata Capacity Optimizer allowed them to reduce their infrastructure spend by nearly 30 percent.

Benefits for Your Team

intro content icon 1

Operation Teams

  • Ensure big data cluster stability and efficiency.
  • Avoid overspending on hardware.
  • Reduce time spent on capacity planning.
  • Run more jobs on existing infrastructure.
intro content icon 2

Application Teams

  • Run more jobs.
  • Access additional cluster capacity.
  • Spend less time in backlog queues.
intro content icon 3

Business Teams

  • Report on capacity trends.
  • Get accurate chargeback reporting.
  • Increase productivity and ROI.

Take a free 15-day trial to see what Big Data success looks like

Pepperdata products provide complete visibility and automation for your big data environment. Get the observability, automated tuning, recommendations, and alerting you need to efficiently and autonomously optimize big data environments at scale.