Looking to further reduce your cloud computing cost? Pepperdata has good news for you: As part of our version 6.3 release, Capacity Optimizer will include managed autoscaling.
Our new innovation to the Pepperdata Analytics Stack Performance Suite adds managed autoscaling to the original functionalities of Capacity Optimizer. This new feature can help your organization reduce up to 50% of your Google Dataproc, Qubole, and Amazon EMR costs.
Autoscaling with Capacity Optimizer
Existing forms of autoscaling do provide the elasticity customers need for their big data workloads. However, they can still lead to over-provisioning and, in turn, runaway costs.
Take a look at how autoscaling typically happens in the image below, which includes two charts. The first chart shows that autoscaling grew the cluster to 100 nodes for the entire runtime duration. However, the second chart shows what the cluster actually ran during the runtime duration. Sometimes it was just one task, other times no tasks were running. The 100 nodes autoscaling provided for the entire duration was, in fact, overkill.