Webinar: Proven Approaches to Hive Query Tuning

Webinar: Proven Approaches to Hive Query Tuning

Looking to further reduce your cloud computing cost? Pepperdata has good news for you: As part of our version 6.3 release, Capacity Optimizer will include managed autoscaling.

Our new innovation to the Pepperdata Analytics Stack Performance Suite adds managed autoscaling to the original functionalities of Capacity Optimizer. This new feature can help your organization reduce up to 50% of your Google Dataproc, Qubole, and Amazon EMR costs.

Autoscaling with Capacity Optimizer

Existing forms of autoscaling do provide the elasticity customers need for their big data workloads. However, they can still lead to over-provisioning and, in turn, runaway costs.

Take a look at how autoscaling typically happens in the image below, which includes two charts. The first chart shows that autoscaling grew the cluster to 100 nodes for the entire runtime duration. However, the second chart shows what the cluster actually ran during the runtime duration. Sometimes it was just one task, other times no tasks were running. The 100 nodes autoscaling provided for the entire duration was, in fact, overkill.

managed autoscaling screenshot to lower cloud computing cost

Capacity Optimizer v6.3 fixes this issue. Intelligently augmenting autoscaling, it will ensure all nodes are fully utilized before additional nodes are created. This then eliminates waste and reduces your Google Dataproc, Qubole, and Amazon EMR costs.

While making thousands of decisions per second, Capacity Optimizer analyzes the resource usage of each node in real time. This allows the solution to optimize the utilization of CPU, memory, and I/O resources on big data clusters. 

Automated deployment options like these from Pepperdata can seamlessly be added to your EMR, Dataproc, and Qubole deployments. Moreover, aside from automatically tuning cloud deployment for optimal performance and reduced cloud computing cost, Pepperdata:

  • Leverages targeted performance insights, reducing troubleshooting time by 90% 
  • Tunes app resources for peak efficiency through prescriptive recommendations
  • Automatically detects and alerts users on SLA-impacting bottlenecks

Pepperdata CEO, Ash Munshi, on managed autoscaling: “Even with the best cloud migration strategy and dedicated attempts to curb costs, the cloud makes managing resources more difficult. But, by leveraging machine learning and managing infrastructure in real time, IT operations teams automatically recapture wasted capacity and significantly reduce their costs.”

When Will Managed Autoscaling Be Available?

The official public release date of Capacity Optimizer v6.3 with managed autoscaling is September 2020. However, companies looking for early access can have the supported beta release with free updates this July. Register for the Capacity Optimizer Beta Program here, and gain the ability to:

  • Reduce troubleshooting time by 90% by leveraging targeted performance insights.
  • Tune application resources for peak efficiency with prescriptive recommendations.
  • Automatically detect and alert on bottlenecks that impact SLAs.