Developers are faced with specific challenges to big data success, including poor performance, unpredictable runtimes, and bottlenecks. They need faster, more efficient applications to meet SLAs, but at times, they submit a query and have no insight into when it starts running — or whether it will finish successfully.
Developers look at different tools to try and understand what’s going on and why things are hung up, but even with all those tools, they don’t have an accurate picture. They have countless questions and have no way to get the answers, which we discussed in our webinar last week:
- Why is my job running slow?
- Is there something wrong with the platform?
- How do I guarantee SLAs of my app?
- How do I maintain my jobs over time?
- How do I make my job go faster?
- How can my job be more efficient?
- Is my job even running?
- Is my job bottlenecked?
To answer these questions, you need self-service access to actionable insights and recommendations. Furthermore, you need tailored alerts and recommendations based on job types, job performance over time, and cluster-wide utilization trends. With most current tools, the metrics to feed these insights aren’t even available – not to you or your infrastructure manager.
That’s where Pepperdata comes in. We create a holistic view of your application performance across the cluster and provide automatic tuning as well as alerts and recommendations for optimal application performance. By that I don’t mean we’re just going to make your application run better this time — we make your application perform best all the time in your multitenant environment. That’s one of the big differences. In the Hadoop space, you can’t just be concerned with how things work this time: You need to tune performance continually within the parameters of the environment you’re dealing with.
Pepperdata Application Spotlight provides all of the tools developers need to optimize performance, including a 360° View, Application Tuning, Application Recommendations, Application Alerts and reporting.
360° View
Sometimes your application performance problems are your fault, and sometimes they’re a result of your multi-tenant environment. You need to be able to look at the last number of times — say 5 times — you ran the application, understand what’s going on in the environment and learn where and why extra time was spent in the queue, to determine whether it’s just you who’s experiencing the slowdown. Maybe others are experiencing similar bottlenecks and if so, your operator needs to know that.
Pepperdata Application Spotlight provides a 360° View of applications in one place — in the context of the entire multi-tenant cluster — giving you quick insight into why your application is running slow. It helps you target your performance tuning by helping you understand exactly what CPU and memory resources it requested, what it needs, what it used, and what it wasted. Pepperdata is the only APM solution that continuously collects metrics about every single application and infrastructure resource, which means you can diagnose application performance issues up to 90% faster and fine-tune each application.
For instance, you get much more insight on what happened than just knowing Spark used a certain amount of memory — you get details into specific jobs and stages, allowing you to drill down to get the why and when so you can make impactful changes. A lot can be going on with your application and understanding where a problem requires time series data so you can do some fine-grained, effective tuning. We provide in-depth insight into that Spark application, unlike other solutions that just provide you with the top layer.
Pepperdata automatically tunes applications that are repetitive for either resource efficiency or runtime, improving performance and efficiency of applications as well as resource utilization. We help you understand and make efficient and cost-effective utilization of your resources by helping you discover where there are inefficiencies or where there are resources that are being over-allocated and even wasteful.
Application Recommendations
Pepperdata also provides job-specific Application Recommendations based on industry best practices and the in-depth knowledge of performance metrics we’ve already discussed. You get self-service recommendations for best queues and launch times — based on current workloads, data partitioning, serialization, container sizes and heap reservations.
Application Alerts
In addition to surfacing performance bottlenecks, Pepperdata improves application runtime predictability by allowing you to create and receive Application Alerts on important events and outcomes so you know when an application is at risk of failure. This alarm can be tailored to go to you or to certain development teams. Use our data to identify any resource bottleneck (CPU, memory, IO, etc.), pinpoint straggling tasks or poor parallelization, and alert on thresholds such as duration, amount of data processed or other milestones.
Reporting
Pepperdata provides you with a holistic picture of everything that’s going on – relevant real-time and historical information about the cluster, including system demand, users, and applications — in one cohesive capacity planning report. This allows you to understand and accurately size queues, gain insight into which applications are using what resources, and plan for future jobs by considering things that tend to go on in your particular environment.
Experts ready to help
We realize you face challenges.I’s not an overstatement to say we’ve probably seen every single issue you face. Our big data experts have been working on most of these problems for a very long time. We’ve seen a lot of systems, a lot of applications and a lot of tasks. So we have a very good understanding of what the average Hadoop environment looks like and what needs to be done to maximize application performance. We’ve worked with many large-cluster customers and have helped them solve a myriad of challenges and problems. It’s likely we’ve seen environments and problems like yours, making us a perfect partner to help you achieve your goals.
Contact sales@pepperdata.com today and learn how we can help you.
More info:
- Watch the webinar on this topic (Developer Q and A – Improve Application Performance and Efficiency).
- Register for our next webinar, Capacity Manager Q and A – How to Improve Productivity, Throughput and Uptime (October 10, 11 am Pacific)
- Request a trial to learn more about how Pepperdata can help you optimize application performance