Application Performance Management for the Big Data Stack

Managing multi-tenant big data clusters is complex. Pepperdata partners with you to deliver predictable performance, empowered users, managed cost, and managed growth with proven big data APM.

View Product

Elements of Big Data APM Success

Pepperdata delivers proven big data APM products, operational experience, and deep expertise.

CUSTOMERS

GEOVANIE MARQUEZ, SOFTWARE ARCHITECT

PHILIPS WELLCENTIVE

“Before Pepperdata, we experimented with various approaches to solve our big data performance issues, but we couldn’t see deep enough into our Hadoop cluster. Platform Spotlight shined a bright light into our Hadoop cluster and provided detailed data that helped us isolate and resolve the problem.”

JESSE ESCOBEDO, SENIOR SYSTEMS ENGINEER

RUBICON PROJECT

“At Rubicon Project, having the appropriate visibility and insight into our big data applications is extremely important when delivering detailed reports to our clients and meeting our SLA. We challenged Pepperdata to find a solution to profile our applications before going to production that would help us maintain our SLA to our customers as we introduce new applications. Pepperdata listened to us and quickly understood the problem we were trying to address.”

MICHAEL MCGOWEN, MANAGER OF DATA ENGINEERING

CHARTBOOST

“Chartboost is the world’s largest mobile games-only advertising platform, reaching one billion active players around the world every month. Chartboost utilizes Apache Spark on large Amazon EC2 Hadoop clusters for machine learning and ETL workflows. Understanding Spark application performance in these complex environments is always a challenge. As a current user of Pepperdata Platform Spotlight, it has been great to work with Pepperdata on the development of the Application Spotlight self-service portal software. It will give us a comprehensive insight into Spark jobs.”

DAVID NGUYEN, SENIOR MANAGER OF EDW OPERATIONS ENGINEERING

EXPEDIA

“The level of support and expertise that we receive from the Pepperdata team made a big difference to us. Pepperdata worked closely with us on our Platform Spotlight implementation to ensure success on our big data cluster. With Pepperdata capacity optimization, the DevOps team runs more jobs, faster. We’ve seen a big performance boost across the cluster and have a much more efficient data footprint. Using the Pepperdata dashboard to see application-level metrics, unique custom views, resource utilization per workload drill-downs, and hardware utilization by various workgroups has significantly improved the way that we manage and troubleshoot big data cluster performance issues.”

Our Customers Get Results

See who’s using Pepperdata big data APM solutions to achieve predictable performance, empower users, and to manage cost and growth for their big data investment.

Customers

Pepperdata Big Data APM Solutions Overview

Evaluating and purchasing a big data APM solution is complicated. We’ve made it easy to understand with our Pepperdata Big Data APM Solutions Overview. Achieving big data performance success has never been easier.

Request a trial to see firsthand how Pepperdata big data solutions can help you achieve big data performance success. Pepperdata’s proven APM solutions provide a 360° degree view of both your platform and applications, with realtime tuning, recommendations, and alerting. See and understand how Pepperdata big data performance solutions helps you to quickly pinpoint and resolve big data performance bottlenecks. See for yourself why Pepperdata’s big data APM solutions are used to manage performance on over 30K Hadoop production clusters.

Request Trial

Resources

Synecdoche and SaaS: What Title Should the Customer Success Leader Have? Image

Pepperdata Announces Major Enterprise-Grade Capabilities, Enhanced Usability and Services

Extensive Enterprise Reporting Capabilities for Application Spotlight and Platform Spotlight and Expanded Professional Services Unveiled at Strata Data NYC

NEW YORK — September 11, 2018 (Strata Data New York 2018, Booth 741) — Pepperdata, the leader in Application Performance Management (APM) for big data success, announced enterprise-grade features to its APM suite that include auto-tuning, enhanced recommendations, and management and operational reporting, powered by an easy-to-use self-service interface. The company also announced professional services offerings that include best-practices, performance planning, capacity planning, and architecture design for big data success.

The company’s new professional services are directly enabled by the vast amount of metrics — 600 trillion data points every year — that Pepperdata collects from tens of thousands of nodes every few seconds. This data provides unique insight into all aspects of operationalizing big data applications. Pepperdata is unique in its ability to deliver not only enterprise-grade software, but also expertise, experience and knowledge that ensures big data success.

“Customers are demanding more than features and function from us — they’re asking us to become partners in making sure their big data investments yield business results,” said Ashfaq Munshi, Pepperdata CEO. “We are the only company offering expert services along with a solution delivering instantaneous time-series data that provides precise insight relevant to enterprise platforms and applications.”

Proven Products

The Pepperdata APM suite — comprised of Platform Spotlight and Application Spotlight — enables tight collaboration between developers and operators, improves overall efficiency and performance, and enables enterprises to do more with their existing big data investments.

Platform Spotlight provides infrastructure and capacity managers with:

  • 360° Platform View: Pepperdata continuously collects exhaustive data in real time about clusters, hosts, queues, users, applications and all relevant resources, providing a single source of operational and performance truth across clusters. This breadth of real-time data, which no other tool or product collects and provides, enables enterprises to quickly diagnose performance issues up to 90% faster than without Pepperdata, while making real-time resource decisions based on user priorities and needs.
  • Real-Time Platform Tuning: Pepperdata increases platform throughput up to 50% by leveraging AI-driven resource management to automatically tune cluster resource usage and recapture wasted capacity.
  • Platform Recommendations: Pepperdata provides actionable reporting and recommendations to rightsize containers, queues and other resources so enterprises can achieve optimal application and cluster performance on multi-tenant systems.
  • Platform Alerting: Pepperdata exposes data at sufficient granularity to avoid nuisance alarms and create tailored alerts that pinpoint the root causes of performance issues and operational inefficiencies.
  • 360° Reports: With its vast amount of data that correlates configuration and tuning changes with changes in platform performance, Pepperdata reports allow executives to understand financial impacts of operational decisions across the platform.

Application Spotlight provides developers with:

  • 360° Application View: Pepperdata provides developers with a holistic source of application performance data within the context of the cluster, and enables them to quickly diagnose issues, reduce troubleshooting time, and improve performance.
  • Application Tuning: Pepperdata provides real-time data from applications and cluster resources, which informs developers’ decisions about application configuration and environment considerations for improving runtime performance. Additionally, Pepperdata automatically tunes applications on an ongoing basis to improve runtime or resource utilization.
  • Application Recommendations: Pepperdata automatically delivers job-specific recommendations based on comparing the values of dozens of performance metrics and tuning parameters using industry heuristics, best practices and in-depth knowledge of those metrics and parameters.
  • Application Alerting: In addition to surfacing performance bottlenecks, Pepperdata enables developers to create and receive alerts about events that degrade application performance so they know when an application is at risk of failure.

Operational Experience and Deep Expertise

Pepperdata continuously monitors over 250 production clusters across its customer base — over 30,000 nodes across all Big Data distributions and hardware configurations — for a total 550 million jobs and 600 trillion data points every year. Coupled with its success serving Fortune 100 customers, this uniquely broad set of data empowers Pepperdata to help customers:

  • Establish and follow best practices and effectively set and achieve strategic initiatives.
  • Stay ahead of the competition by providing faster applications and more efficient resource usage.
  • Stay ahead of capacity needs and squeeze the most out of existing capacity.
  • Design a successful architecture using real-world experience derived from some of the world’s biggest clusters.
  • Successfully support developers and operations managers by providing self-service access to data-rich, curated, self-service portals.
  • Pepperdata will be exhibiting at the Strata Data Conference at the Jacob Javits Center (booth 741) in New York City, September 12th and 13th.

Helpful Links

About Pepperdata

Pepperdata is the leader in Application Performance Management solutions and services for big data success, solving application and platform issues throughout the stack for developers as well as capacity and infrastructure managers. The company partners with its customers to provide proven products, operational experience, and deep expertise to deliver predictable performance, empowered users, managed costs and managed growth for their big data investments, both on-premise and in the cloud. Leading companies like Comcast, Philips Wellcentive and NBC Universal depend on Pepperdata to deliver big data success.

Founded in 2012 and headquartered in Cupertino, California, Pepperdata has attracted executive and engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata investors include Costanoa Ventures, Signia Venture Partners, Silicon Valley Data Capital and Wing Venture Capital, along with leading high-profile individual investors. For more information, visit www.pepperdata.com.

###

Contact:
Samantha Leggat
samantha@pepperdata.com

Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.

September 11, 2018
Synecdoche and SaaS: What Title Should the Customer Success Leader Have? Image

Pepperdata Announces Executive Appointments and the Close of Recent 
Funding Round

Pepperdata Anticipates Doubling Team Within a Year to Support Increase in Seven-Figure Sales Deals

CUPERTINO, Calif. — September 4, 2018 — Pepperdata, the leader in Application Performance Management (APM) for big data success, announced the appointment of two executives reporting to CEO Ash Munshi. The appointments include Charles Marker as Vice President of Engineering and Dan Marx as Vice President of Sales. This announcement comes on the heels of the company’s latest funding, which Pepperdata will allocate to hiring and product development as it continues to deliver on feature requests from customers to support their mission-critical big data deployments.

Charles Marker joins Pepperdata as VP of Engineering from his previous position as Global Head of Engineering at Guidewire Software. Prior to Guidewire, Mr. Marker held Engineering VP positions at Kontagent, Yahoo, Qualcomm and Atheros. Dan Marx, who has been with Pepperdata since 2014, has been named VP of Sales. Mr. Marx brings deep expertise and experience in enterprise sales, including extensive success in big data technology sales at WANdisco and Zettaset.

“We are excited about the contagious enthusiasm and deep expertise Charles and Dan bring to Pepperdata,” said Mr. Munshi, Pepperdata CEO. “As we close more and more seven-figure deals, we are pleased to have the funding necessary to make appointments like these, and we will continue expanding to support the tremendous growth we’re experiencing. We anticipate more than doubling our team within a year.”

“Pepperdata is the leader in Application Performance Management for big data, delivering scalable solutions that enable Fortune 100 companies to achieve successful outcomes from their investments. We continue to be impressed with Pepperdata’s ability to facilitate adoption by these leading companies by identifying use cases that benefit from APM. We are pleased to work with them as they continue to execute their strategy,” said Jim McLean, Managing Director at Silicon Valley Data Capital.

“We were impressed to see the world’s biggest and best AI-driven companies already using Pepperdata so their Hadoop and Spark clusters perform at scale. Pepperdata helps ensure these global brands in e-commerce, voice applications and consumer banking optimize both the productivity and performance of their big data practices. Their continued team and company growth is exciting,” said Greg Sands, Managing Partner at Costanoa Ventures.

Since its founding in 2012, Pepperdata has established itself as a leader in APM for big data success, delivering proven products, operational experience, and deep expertise for its customers. Pepperdata is deployed at Fortune 100 companies in financial services, retail, healthcare, telecommunications and more, totaling more than 250 production clusters with 30,000 nodes spanning all big data distributions and hardware configurations. With the level of data the company collects — over 550 million jobs and 600 trillion data points annually — and its extensive global enterprise experience, Pepperdata is the wise choice for companies looking to get more value and optimal performance from their big data investments.

Pepperdata will be exhibiting at the Strata Data Conference at the Jacob Javits Center (booth 741) in New York City, September 12th and 13th.

Helpful Links

About Pepperdata

Pepperdata is the leader in Application Performance Management solutions and services for big data success, solving application and platform issues throughout the stack for developers as well as capacity and infrastructure managers. The company partners with its customers to provide proven products, operational experience, and deep expertise to deliver predictable performance, empowered users, managed costs and managed growth for their big data investments, both on-premise and in the cloud. Leading companies like Comcast, Philips Wellcentive and NBC Universal depend on Pepperdata to deliver big data success.

Founded in 2012 and headquartered in Cupertino, California, Pepperdata has attracted executive and engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata investors include Costanoa Ventures, Signia Venture Partners, Silicon Valley Data Capital and Wing Venture Capital, along with leading high-profile individual investors. For more information, visit www.pepperdata.com.

###

Contact:

Samantha Leggat

samantha@pepperdata.com

Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.

September 4, 2018
Synecdoche and SaaS: What Title Should the Customer Success Leader Have? Image

Pepperdata Announces Application Spotlight–New Self-Service APM Portal

CUPERTINO, Calif.March 1, 2018Pepperdata, the Big Data Performance company that provides developer and operator solutions to optimize application and cluster performance in Big Data environments, announces Application Spotlight. This self-service portal enables Big Data application developers to generate application-specific recommendations to improve application performance, highlight applications that need attention, automatically identify bottlenecks, and alert on duration, failure conditions, and resource usage.

Application Spotlight helps developers to search for all applications running on the cluster or just the specific applications that they are interested in, compare current and previous runs, and visualize Spark applications and its stages for easy root cause failure analysis and performance tuning.

“Application Spotlight provides relevant application information, insights, and calls to action, all in one place, so that developers can easily and quickly perform these tasks,” said Ash Munshi, CEO of Pepperdata. “In addition to helping developers make jobs go faster, Application Spotlight enables them to be better tenants in multi-tenant clusters by showing them how to write optimal jobs and more efficiently use their queue and cluster resources with practical, innovative application performance management solutions. Application Spotlight enables developers to quickly understand performance impacts and get recommendations on how to better optimize their jobs.”

Big Data Performance Solutions for Both Developers and Operators

The Pepperdata product suite provides solutions for monitoring, tuning, troubleshooting, applications and clusters along with automated cluster optimization. Pepperdata allows enterprises to:

  • Reduce time-to-problem resolution using comprehensive and detailed performance data, allowing developers and operators to troubleshoot performance problems 10x faster.
  • Provide developers with solutions that help them identify and fix application performance problems caused by excessive usage of resources and application errors
  • Automatically increase cluster capacity utilization by 30 percent to 50 percent without adding hardware.

Pepperdata products improve collaboration between development and operations teams by providing both an application as well as cluster view of performance. Pepperdata products, are used to monitor and manage mixed workloads from frameworks such as Spark, MapReduce, Kafka, Tez, Solr, and Impala.

See a Demo of Application Spotlight at Strata San Jose

Pepperdata is sponsoring, speaking, and exhibiting at Strata Strata Data Conference on March 6–8 at the San Jose Convention Center. Please visit Pepperdata Booth to discuss your requirements and see a demonstration of Pepperdata Application Spotlight. For information about our Strata schedule, go to www.pepperdata/events.

Helpful Links

About Pepperdata

Pepperdata is the big data performance company. Leading companies such as Comcast, Philips Wellcentive, and NBC Universal depend on Pepperdata to manage and improve the performance of Hadoop and Spark. Enterprise customers use Pepperdata products and services to troubleshoot performance problems in production, increase cluster utilization, and enforce policies to support multi-tenancy. Pepperdata products and services work with customer Big Data systems both on-premise and in the cloud.

Founded in 2012, Pepperdata has raised $20M from investors including Citi Ventures, Signia Venture Partners and Wing Venture Capital, and attracted senior engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata is headquartered in Cupertino, California. For more information, visit pepperdata.com.

###

Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.

Media Contact

Jim Dvorak
Offleash for Pepperdata
(415) 735-1622
pepperdata@offleashpr.com

March 1, 2018
Synecdoche and SaaS: What Title Should the Customer Success Leader Have? Image

Ensuring Uptime and Performance for Life-Saving Healthcare Applications — Clearsense and Pepperdata  

Ensuring Uptime and Performance for Life-Saving Healthcare Applications — Clearsense and Pepperdata  

Hospitals, physician groups, insurance providers, and research organizations rely on the Clearsense Platform to support life-saving applications, such as predictive algorithms for cardiac arrest and sepsis that notify caregivers and healthcare providers that such an event may occur. Other algorithms help hospital clients reduce costs and improve supply chain management.

To ensure 99.999% uptime for these critical applications, Clearsense needed a scalable, reliable and comprehensive big data APM solution. According to Charles Boicey, Clearsense CIO, “There is no tolerance for downtime in healthcare, which is why we bought Pepperdata.”

In addition to ensuring uptime, effective capacity planning is also vital to the Clearsense Platform. “It’s essential that we run at maximum efficiency and plan accurately for growth to avoid disaster,” said Boicey. “Instead of just throwing servers at the platform, which is costly, we can effectively manage growth because Pepperdata provides insights on how products and queries are affecting the environment. Without Pepperdata, we don’t have the insights we need to scale sufficiently and safely.”

Clearsense is a pioneer of the data revolution, and its partnership with Pepperdata is key to its mission. Says Boicey, “Together, Clearsense and Pepperdata empower healthcare organizations to harness the full power of data to deliver the best possible medical, financial and operational outcomes.”

The Pepperdata APM solution provides Clearsense with the following:

To learn more, download the complete Clearsense Case Study (PDF). 

Additional Resources

  • Clearsense – learn more about Clearsense, providing healthcare organizations with the ability to gain insights by analyzing all of the data from their disparate sources to improve patient outcomes, increase revenue, create opportunities, and lower costs.
  • Download the complete Clearsense Case Study (PDF)

Watch our webinar with Clearsense.

December 11, 2018
Synecdoche and SaaS: What Title Should the Customer Success Leader Have? Image

Why is My App Running so Slow

Determining exactly what causes a performance problem in a big data environment is a complex challenge. The first step is identifying the root cause, which is not exactly straightforward. Here are a few potential issues and ideas developers can use to fix them.

When you can affect change

Code is not optimal

As a developer, you can improve the performance of an application by modifying your code. Whether this is optimizing your SQL for Hive, or using the correct IO calls for the type of data you are retrieving, this is one of the areas you can make a direct impact on your application’s performance. There are tools that can help with both, including Pepperdata.

Dataset skew (distribution/keyspace)

Sometimes you can fix this, sometimes you can’t — it depends on how much control you have on your incoming data. But if the dataset is very skewed, parts of your application can easily take 2-3 times longer than necessary processing your data as other parts.

You’re asking for incorrect resources

This one is slightly more subtle, and can change based on the amount of data you are processing, but one of the easiest ways to improve application performance in Hadoop is to ask for the correct amount of resources — not too much and not too little. If you are asking for your application to use 50TB of memory and 2000 cores (1000 tasks each asking for 5GB and 2 cores), but you only use 40 percent of that at peak use (largest task is 2GB and 1 core), then you are impacting two things:

  • your performance, as you will need to wait for more space in the queue to launch your application, and
  • the performance of other tenants on the system, as you are occupying queue space they could be using when you aren’t using the physical resources.

When you cannot affect change because it’s not your fault

Contention for queue resources

If you are assigned to a particular set of queues, you should submit your application to one of those. However, if those queues are full, your application could:

  • Spend more time in queue than executing, or
  • Be limited to running only one or two tasks at a time as opposed to taking advantage of the parallel compute capability of the Hadoop platform.

The only way to fix this issue is to find time to run your applications when the queue is less busy, or talk to your system administrator about your own queue if you have SLA’s around your applications.

Contention for physical resources

Sometimes, there are actual physical resource bottlenecks you will run into, even if your application is doing everything correctly and is able to fully utilize the queue you are assigned to. This is difficult to diagnose, as Hadoop will not expose this by default. Your system operators should have tooling to let them look at the system resource utilization during the time window that your application is running, and see what the resource contention is.

  • Is your application running on cluster nodes where the CPU utilization is above 95 percent? This is not uncommon in busy environments, and depending on the version of Hadoop and the scheduler your system is using, there may not be any controls preventing this from happening!
  • Is someone else using all of the IO bandwidth? This isn’t a resource you can request and use for scheduling (unless your company is using MapR), so it is possible for one application to use all of the IO bandwidth that should be being shared between 10-100 applications are any given moment.

Learn more:

November 27, 2018
Synecdoche and SaaS: What Title Should the Customer Success Leader Have? Image

Possibilities for Big Data in Healthcare are Endless

There are more reasons than ever to be optimistic about the possibilities created by big data and predictive analytics in healthcare. By leveraging new technologies, including big data APM solutions like Pepperdata, healthcare organizations are able to make better medical and financial decisions while continually improving the quality of patient care. For instance, our customer, Clearsense, enables its healthcare customers to leverage analytics to save lives and improve patient outcomes.

The use cases made possible with big data and predictive analytics are truly visionary and life-saving. Here are some examples.

Saving Lives

  • The U.S. government has instituted patient assessments for the risk of overdose and suicide, enabling the effective collaboration between medical and mental care providers serving veterans at risk of suicide (White House blog post).  
  • AI can solve the problem associated with the shortage of trained healthcare providers, which can significantly limit access to life-saving care in developing nations. According to an example cited in Health IT Analytics,  “AI imaging tools can screen chest x-rays for signs of tuberculosis, often achieving a level of accuracy comparable to humans” making this capability available to providers in low-resource areas and reducing the need for an on-site specialist.
  • UC Davis uses EHR data in an algorithm that gives clinicians an early warning about sepsis, which has a 40 percent mortality rate and is difficult to detect until it’s too late.

Improving Outcomes

  • According to Science Direct, King Faisal Specialist Hospital and Research Center utilized analytics to identify areas of emergency room (ER) deficiency and uncover ways to improve performance. By collecting and analyzing data on the length of stay for patients (efficiency) and the percentage of patients leaving without treatment (effectiveness), the organization implemented new programs to improve outcomes, including segmenting lower severity patients and establishing an internal waiting room.
  • The Health Workforce Mapper tool from the American Medical Association (AMA) offers reliable data from AMA, CMS, and the CDC, enabling users to identify workforce-related trends and understand which socioeconomic factors significantly impact patients’ access to healthcare.

Personalizing Care

Lowering Costs

  • Organizations spend billions every year on unplanned hospital readmissions, which are largely avoidable. Using patient data, hospitals can uncover factors that raise the risk of readmission and design preventive measures to avoid these costly occurrences, such as implementing nutrition interventions and educational programs to improve patient outcomes.
  • In one study published in American Health and Drug Benefits, a nutrition-focused quality improvement program led to a reduction in 30-day hospital readmissions and length of stay, resulting in significant per-patient net savings.

Managing Population Health

While big data can be difficult to manage and process, it will undoubtedly continue to transform the healthcare industry in a positive way. By leveraging new technologies, including Pepperdata, healthcare organizations are gleaning highly useful and actionable information and enabling caregivers and administrators to make better medical and financial decisions while delivering an ever-increasing quality of patient care.

Register for our webinar to learn how Clearsense enables healthcare organizations to consolidate, access and visualize data to save lives, improve patient outcomes, reduce costs, and avoid waste.

November 13, 2018