Continuously Improve Capacity Utilization of Production Clusters Without Manual Tuning or Intervention

Enabled in Platform Spotlight, Capacity Optimizer runs continuously to improve the capacity utilization of existing production clusters without manual tuning or intervention. 

Pepperdata Capacity Optimizer automatically tunes and optimizes your cluster resources, recapturing wasted capacity so you can run more applications and get the most out of your infrastructure investment.

This video explains how Capacity Optimizer enables you to achieve optimal hardware performance in your multi-tenant distributed computing environment.

Capacity Optimizer makes thousands of decisions per second, increasing typical enterprise cluster throughput by up to 50 percent.

Improve Cluster Throughput Up to 50%

By monitoring the entire infrastructure in real time, including hardware and applications, and leveraging AI with active resource management, Pepperdata Capacity Optimizer identifies where more work can be done and adds tasks to servers with available resources.

On a typical cluster, Capacity Optimizer makes thousands of decisions per second, increasing typical enterprise cluster throughput by up to 50 percent. Even the most experienced operator dedicated to resource management can’t make manual configuration changes with the precision and speed of Capacity Optimizer.

Doesn’t YARN Scheduler Manage Resources?

YARN (“Yet Another Resource Negotiator”) scheduler leverages the resource management capabilities of MapReduce, coordinating consumption and usage reservations to make allocations. Limited by its conservative assumptions about memory usage, YARN under-provisions resources. In addition, YARN does not monitor containers once they start running or adjust in real time based on actual usage.

Pepperdata Capacity Optimizer overcomes these limitations and maximizes resource utilization by monitoring actual per-task hardware usage in real time and dynamically making adjustments at the process level, eliminating inefficiencies and bottlenecks, and maximizing resource usage.

Pepperdata Capacity Optimizer leverages active resource management features in Hadoop to dynamically tune cluster resources and eliminate inefficiencies and bottlenecks.



  • Maximize your infrastructure investment

  • Achieve a 30-50 percent increase in throughput performance

  • Ensure cluster stability and efficiency

  • Avoid overspending on unnecessary hardware

  • Reduce time spent on capacity planning

  • Run more jobs concurrently on your existing infrastructure


  • Run more jobs

  • Access additional cluster capacity

  • Spend less time in backlog queues

  • Run more jobs concurrently on your existing infrastructure

Enterprise Organizations

  • Eliminate inefficiencies and bottlenecks to ensure infrastructure stability

  • Automatically tune applications

  • Recapture wasted capacity to optimize cluster resources

  • Run more applications

  • Get the most out of your infrastructure investment

  • Run more jobs concurrently on your existing infrastructure

Pepperdata Shines a Light Into Big Data Healthcare Cluster

Philips Wellcentive fixed the root cause of a critical performance issue within days of installing Platform Spotlight. Their Hadoop cluster achieved a significant increase in cluster performance throughput. Jobs completed on time and SLAs were met. The technical team returned to focusing on their core initiatives, streamlining operations, and meeting revenue targets.

Request a trial to see firsthand how Pepperdata big data solutions can help you achieve big data performance success. Pepperdata’s proven APM solutions provide a 360° degree view of both your platform and applications, with realtime tuning, recommendations, and alerting. See and understand how Pepperdata big data performance solutions helps you to quickly pinpoint and resolve big data performance bottlenecks. See for yourself why Pepperdata’s big data APM solutions are used to manage performance on over 30K Hadoop production nodes.

Request Trial