If you’ve ever had to contend with bottlenecked Hadoop jobs or with missing a critical service-level agreement (SLA) because of sluggish runtimes, you understand the importance of having your Hadoop cluster perform predictably. This is the heart of what Pepperdata software is all about: ensuring quality of service (QoS) for your Hadoop applications.
As such, we’re thrilled to have gotten a shout out from Senior Analyst Mike Matchett at the Taneja Group in his latest article for Tech Target: Can Your Cluster Management Tools Pass Muster? The article explores the challenges of managing multi-tenant, multi-workload clusters, and Mike zeroes in on what we believe is the most salient issue facing these sorts of Hadoop deployments: ensuring resources are allocated intelligently so that mission-critical jobs always complete on time. He writes,
But the real trick is performance management, the key to which is knowing who’s doing what, and when. At a minimum, there are standard tools that can generate reports out of the (often prodigious) log files collected across a cluster. But this approach gets harder as log files grow. And when it comes to operational performance, what you really need is to optimize QoS and runtimes for mixed-tenant and mixed-workload environments. For example, Pepperdata assembles a live run-time view of what’s going on across the cluster, and then uses that insight to dynamically control the assignment of cluster resources. This assures priority applications meet service-level agreements while minimizing needed cluster infrastructure.