Managing multi-tenant big data clusters is complex. Pepperdata partners with you to deliver predictable performance, empowered users, managed cost, and managed growth with proven big data APM.
Pepperdata delivers proven big data APM products, operational experience, and deep expertise.
PHILIPS WELLCENTIVE
“Before Pepperdata, we experimented with various approaches to solve our big data performance issues, but we couldn’t see deep enough into our Hadoop cluster. Platform Spotlight shined a bright light into our Hadoop cluster and provided detailed data that helped us isolate and resolve the problem.”
RUBICON PROJECT
“At Rubicon Project, having the appropriate visibility and insight into our big data applications is extremely important when delivering detailed reports to our clients and meeting our SLA. We challenged Pepperdata to find a solution to profile our applications before going to production that would help us maintain our SLA to our customers as we introduce new applications. Pepperdata listened to us and quickly understood the problem we were trying to address.”
CHARTBOOST
“Chartboost is the world’s largest mobile games-only advertising platform, reaching one billion active players around the world every month. Chartboost utilizes Apache Spark on large Amazon EC2 Hadoop clusters for machine learning and ETL workflows. Understanding Spark application performance in these complex environments is always a challenge. As a current user of Pepperdata Platform Spotlight, it has been great to work with Pepperdata on the development of the Application Spotlight self-service portal software. It will give us a comprehensive insight into Spark jobs.”
EXPEDIA
“The level of support and expertise that we receive from the Pepperdata team made a big difference to us. Pepperdata worked closely with us on our Platform Spotlight implementation to ensure success on our big data cluster. With Pepperdata capacity optimization, the DevOps team runs more jobs, faster. We’ve seen a big performance boost across the cluster and have a much more efficient data footprint. Using the Pepperdata dashboard to see application-level metrics, unique custom views, resource utilization per workload drill-downs, and hardware utilization by various workgroups has significantly improved the way that we manage and troubleshoot big data cluster performance issues.”
See who’s using Pepperdata big data APM solutions to achieve predictable performance, empower users, and to manage cost and growth for their big data investment.
Evaluating and purchasing a big data APM solution is complicated. We’ve made it easy to understand with our Pepperdata Big Data APM Solutions Overview. Achieving big data performance success has never been easier.
Request a trial to see firsthand how Pepperdata big data solutions can help you achieve big data performance success. Pepperdata’s proven APM solutions provide a 360° degree view of both your platform and applications, with realtime tuning, recommendations, and alerting. See and understand how Pepperdata big data performance solutions helps you to quickly pinpoint and resolve big data performance bottlenecks. See for yourself why Pepperdata’s big data APM solutions are used to manage performance on over 30K Hadoop production clusters.
Pepperdata Application Spotlight Free for Enterprises with a Single Cluster with Up to 20 Nodes
NEW YORK — January 8, 2019 — Pepperdata, the leader in big data Application Performance Management (APM), today announced a free version of Application Spotlight™ for enterprises for use in a single cluster with up to 20 nodes.
Pepperdata Application Spotlight is a self-service APM solution that provides developers with a holistic and real-time view of their applications in the context of the entire big data cluster, allowing them to quickly identify and fix problems (failed Spark applications, for instance) to improve application runtime, predictability, performance and efficiency. Pepperdata Application Spotlight also provides automatic tuning for applications, delivers job-specific recommendations, and enables users to set up alerts on specific behaviors and outcomes to avoid the risk of failure.
“This free offering gives developers a powerful introduction to the robust capabilities of Pepperdata,” said Ash Munshi, Pepperdata CEO. “Application Spotlight gives them self-service access to all the data on their applications in one place so they can determine whether performance issues were caused by their application or other applications running on the cluster.”
With Pepperdata Application Spotlight, developers can:
Availability
Application Spotlight is available immediately for free for use in a single cluster with up to 20 nodes by visiting www.pepperdata.com/free-enterprise-apm.
About Pepperdata
Pepperdata (https://pepperdata.com) is the leader in big data Application Performance Management solutions and services, solving application and infrastructure issues throughout the stack for developers and operations managers. The company partners with its customers to provide proven products, operational experience, and deep expertise to deliver predictable performance, empowered users, managed costs and managed growth for their big data investments, both on-premise and in the cloud. Leading companies like Comcast, Philips Wellcentive and NBC Universal depend on Pepperdata to deliver big data success. Founded in 2012 and headquartered in Cupertino, California, Pepperdata has attracted executive and engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata investors include Citi Ventures, Costanoa Ventures, Signia Venture Partners, Silicon Valley Data Capital and Wing Venture Capital, along with leading high-profile individual investors. For more information, visit www.pepperdata.com.
###
Contact:
Samantha Leggat
samantha@pepperdata.com
Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata. Other names may be trademarks of their respective owners.
Extensive Enterprise Reporting Capabilities for Application Spotlight and Platform Spotlight and Expanded Professional Services Unveiled at Strata Data NYC
NEW YORK — September 11, 2018 (Strata Data New York 2018, Booth 741) — Pepperdata, the leader in Application Performance Management (APM) for big data success, announced enterprise-grade features to its APM suite that include auto-tuning, enhanced recommendations, and management and operational reporting, powered by an easy-to-use self-service interface. The company also announced professional services offerings that include best-practices, performance planning, capacity planning, and architecture design for big data success.
The company’s new professional services are directly enabled by the vast amount of metrics — 600 trillion data points every year — that Pepperdata collects from tens of thousands of nodes every few seconds. This data provides unique insight into all aspects of operationalizing big data applications. Pepperdata is unique in its ability to deliver not only enterprise-grade software, but also expertise, experience and knowledge that ensures big data success.
“Customers are demanding more than features and function from us — they’re asking us to become partners in making sure their big data investments yield business results,” said Ashfaq Munshi, Pepperdata CEO. “We are the only company offering expert services along with a solution delivering instantaneous time-series data that provides precise insight relevant to enterprise platforms and applications.”
Proven Products
The Pepperdata APM suite — comprised of Platform Spotlight and Application Spotlight — enables tight collaboration between developers and operators, improves overall efficiency and performance, and enables enterprises to do more with their existing big data investments.
Platform Spotlight provides infrastructure and capacity managers with:
Application Spotlight provides developers with:
Operational Experience and Deep Expertise
Pepperdata continuously monitors over 250 production clusters across its customer base — over 30,000 nodes across all Big Data distributions and hardware configurations — for a total 550 million jobs and 600 trillion data points every year. Coupled with its success serving Fortune 100 customers, this uniquely broad set of data empowers Pepperdata to help customers:
Helpful Links
About Pepperdata
Pepperdata is the leader in Application Performance Management solutions and services for big data success, solving application and platform issues throughout the stack for developers as well as capacity and infrastructure managers. The company partners with its customers to provide proven products, operational experience, and deep expertise to deliver predictable performance, empowered users, managed costs and managed growth for their big data investments, both on-premise and in the cloud. Leading companies like Comcast, Philips Wellcentive and NBC Universal depend on Pepperdata to deliver big data success. Founded in 2012 and headquartered in Cupertino, California, Pepperdata has attracted executive and engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata investors include Costanoa Ventures, Signia Venture Partners, Silicon Valley Data Capital and Wing Venture Capital, along with leading high-profile individual investors. For more information, visit www.pepperdata.com.
###
Contact:
Samantha Leggat
samantha@pepperdata.com
Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.
Pepperdata Anticipates Doubling Team Within a Year to Support Increase in Seven-Figure Sales Deals
CUPERTINO, Calif. — September 4, 2018 — Pepperdata, the leader in Application Performance Management (APM) for big data success, announced the appointment of two executives reporting to CEO Ash Munshi. The appointments include Charles Marker as Vice President of Engineering and Dan Marx as Vice President of Sales. This announcement comes on the heels of the company’s latest funding, which Pepperdata will allocate to hiring and product development as it continues to deliver on feature requests from customers to support their mission-critical big data deployments.
Charles Marker joins Pepperdata as VP of Engineering from his previous position as Global Head of Engineering at Guidewire Software. Prior to Guidewire, Mr. Marker held Engineering VP positions at Kontagent, Yahoo, Qualcomm and Atheros. Dan Marx, who has been with Pepperdata since 2014, has been named VP of Sales. Mr. Marx brings deep expertise and experience in enterprise sales, including extensive success in big data technology sales at WANdisco and Zettaset.
“We are excited about the contagious enthusiasm and deep expertise Charles and Dan bring to Pepperdata,” said Mr. Munshi, Pepperdata CEO. “As we close more and more seven-figure deals, we are pleased to have the funding necessary to make appointments like these, and we will continue expanding to support the tremendous growth we’re experiencing. We anticipate more than doubling our team within a year.”
“Pepperdata is the leader in Application Performance Management for big data, delivering scalable solutions that enable Fortune 100 companies to achieve successful outcomes from their investments. We continue to be impressed with Pepperdata’s ability to facilitate adoption by these leading companies by identifying use cases that benefit from APM. We are pleased to work with them as they continue to execute their strategy,” said Jim McLean, Managing Director at Silicon Valley Data Capital.
“We were impressed to see the world’s biggest and best AI-driven companies already using Pepperdata so their Hadoop and Spark clusters perform at scale. Pepperdata helps ensure these global brands in e-commerce, voice applications and consumer banking optimize both the productivity and performance of their big data practices. Their continued team and company growth is exciting,” said Greg Sands, Managing Partner at Costanoa Ventures.
Since its founding in 2012, Pepperdata has established itself as a leader in APM for big data success, delivering proven products, operational experience, and deep expertise for its customers. Pepperdata is deployed at Fortune 100 companies in financial services, retail, healthcare, telecommunications and more, totaling more than 250 production clusters with 30,000 nodes spanning all big data distributions and hardware configurations. With the level of data the company collects — over 550 million jobs and 600 trillion data points annually — and its extensive global enterprise experience, Pepperdata is the wise choice for companies looking to get more value and optimal performance from their big data investments.
Pepperdata will be exhibiting at the Strata Data Conference at the Jacob Javits Center (booth 741) in New York City, September 12th and 13th.
Helpful Links
About Pepperdata
Pepperdata is the leader in Application Performance Management solutions and services for big data success, solving application and platform issues throughout the stack for developers as well as capacity and infrastructure managers. The company partners with its customers to provide proven products, operational experience, and deep expertise to deliver predictable performance, empowered users, managed costs and managed growth for their big data investments, both on-premise and in the cloud. Leading companies like Comcast, Philips Wellcentive and NBC Universal depend on Pepperdata to deliver big data success.
Founded in 2012 and headquartered in Cupertino, California, Pepperdata has attracted executive and engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata investors include Costanoa Ventures, Signia Venture Partners, Silicon Valley Data Capital and Wing Venture Capital, along with leading high-profile individual investors. For more information, visit www.pepperdata.com.
###
Contact:
Samantha Leggat
Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.
During the course of administering a Hadoop Cluster, there is often a time when you need to determine what application is causing the current performance degradation. This could be an application that starts and begins exhibiting bad behavior immediately, or it could be a long running ETL job or Spark ML job that suddenly starts using all of the resources of the nodes it is running on.
What now? You know there is an issue on the cluster, but you need to be able to do a few things in order to fix it:
Remediate: If this is a one-time bad behavior caused by a massive shift in data-set, work with the team involved to insure you are aware of future such events. If the application has consistently performed poorly over time, but has finally been identified as the culprit, it might be necessary to ask for a refactor. Quantifying the resource usage will be very important as part of this ask.
Pepperdata makes all three of these very easy to do.
Following is a view showing us that 4 worker nodes in the cluster were pushed to 98-99% User CPU. This is going to cause many problems if left unchecked:
As you can see below, this Spark GraphX application was using over 98 percent of the User CPU on the four worker nodes in this cluster. There were also Hive queries running at the same time, and the long running ones in this chart see their share of CPU plummet.
Now that you have identified the issue, found the culprit, and quantified both what the application was doing and its effect on the system, you need to move on to remediation. Pepperdata helps here both in providing the identification and quantification, but also in providing recommendations, and in the case of Spark, a view of the internals, therefore identifying which stage of a Spark application was at fault and allowing the developer to understand where in their code to start looking for the issue to make changes.
This granularity and ease of discovery exists everywhere in Pepperdata: IO, Physical Memory, Queue memory, CPU, File opens, Sockets, etc. When an application is causing an issue in your environment, whether it is causing Namenode RPC timeouts, completely clogging a capacity-scheduled queue, or writing multiple petabytes into HDFS, we can help you identify the issue immediately and give you the information you need to take action. You can also set alerts on specific parameters to catch repeat offenders, or to catch applications that cause bottlenecks.
Why not try Pepperdata APM for free?
More info:
Can you believe that humanity has created 90% of today’s data within the last two years alone, at the rate of 2.5 quintillion bytes of data per day? While this data is mainly derived from the internet, including social media, web searches, text messages, and media files, IoT devices and sensors are becoming a significant source of data as well. These are the key drivers for the global big data market, which reached $42 billion dollars in 2018, and is forecast by Statista/Wikibon to grow to $64 billion by 2021.
Big Data market size, based on revenue, from 2011 to 2027 ($U.S. billion)
Source: Wikibon; SiliconANGLE, Statista, 2019
Big data is a key driver of overall growth in stored data. According to Cisco’s latest Cloud Global Index report, the volume of stored big data will reach 403 Exabytes (EB) by 2021, up almost 8-fold from 51 EB in 2016. Big data alone will represent 30 percent of data stored in data centers by 2021, up from 18 percent in 2016. Big data is defined here as data deployed in a distributed processing and storage environment, such as Hadoop, Spark or NoSQL clusters. SQL-on-Hadoop engines from vendors like IBM and Oracle can support massive databases have grown in popularity, augmenting the growth of the big data ecosystem.
So big data is, well, bigger than ever. The world is powered by big data and will be for the foreseeable future. But the relentless growth in big data is like a double-edged sword. While businesses are deriving tremendous insights from being able to analyze large data sets, development teams are dealing with more resource-hungry workloads. Because of these related pressures, application performance management (APM) has become an essential element in today’s big data ecosystem. An examination of big data deployments in key verticals shows us what’s keeping developers busy and why APM is a must-have.
The banking industry is leading the growth in big data and business analytics services, whose revenues will surpass $205 billion by 2020, according to the European Banking Federation. Banks are embracing data scientists and integrating advanced tools running on artificial intelligence and cloud computing to structure data more efficiently. Financial services that are enhanced and benefit from data analytics are endless. Think of lending, saving, investing, compliance, fraud monitoring, due diligence, customer insights, anti-money laundering, credit scoring and payments. For example, since credit cards produce so much data and can quickly fall into the wrong hands, fraud has become rampant. Big Data and machine learning are helping to police this illegal activity, stopping fraud before it even starts. In many cases, you’ll receive a notification on your smartphone asking if you are indeed the one making the purchase. If not, then you can halt the transaction and start the process of regaining your financial privacy and security.
Controlling health care costs while striving to improve the quality of care is the greatest challenge facing the healthcare sector. A recent survey from Black Book Research found that 93% of hospital and physician financial executives are actively seeking ways to use big data analytics to link care with and patient outcomes. To that end, many are using big data analytics as a foundation for clinical documentation improvement (CDI). CDI is the process of enhancing medical data collection to maximize claims reimbursement revenue and improve care quality. It’s considered to be a fundamental cornerstone for data quality, accurate reporting, fraud reduction, and robust public health information tracking. And organizations like the College of Healthcare Information Management Executives (CHIME) and the Healthcare Information and Management Systems Society (HIMSS) continue to be focused on achieving better health care through information and technology.
We all have a digital footprint and believe it or not, almost everything we do online can be analyzed, quantified and used to help track consumer trends and behaviors and develop insights that help retailers reach out to us on an engaging, personal level. By understanding big data-based insights on customer habits, retailers can understand which of their products and services are most in-demand and which ones they should potentially stop offering. This not only helps reduce overhead, but guides retailers where to place investment and helps them give the consumer exactly what they want. Trend forecasting algorithms in big data are also helping retailers make key market predictions and forecast consumer trends. By gaining access to insights on real-time customer transactions, retailers get a better understanding which prices yield the best results on particular products. They are also better able to manage their supply-chain logistics. Retail giant Walmart has reaped the rewards of real-time merchandising. Big data technology can also be utilized for “markdown optimization”, an understanding of when prices on particular items should be dropped.
While these trends reflect advances in data analytics and remarkable growth in adoption, they also mean greater challenges for development teams responsible for managing applications, scheduling jobs, and driving analytics initiatives for their organizations. Developers need to clearly understand the performance metrics of their applications to ensure SLAs, avoid failure, improve efficiency, and monitor resource capacity.
Pepperdata Application Spotlight™ is a self-service APM solution that provides developers with a holistic and real-time view of their applications in the context of the entire big data cluster, allowing them to quickly identify and fix problems (failed Spark applications, for instance) to improve application runtime, predictability, performance and efficiency. Application Spotlight also provides automatic tuning for applications, delivers job-specific recommendations, and enables users to set up alerts on specific behaviors and outcomes to avoid the risk of failure.
Now you can experience Pepperdata’s industry-leading APM solution, Application Spotlight, for free. It’s available for enterprises for use in with a single cluster with up to 20 nodes. Get the free version of Application Spotlight by visiting www.pepperdata.com/free-enterprise-apm.
It’s a new year, and time to turn our attention to what the future may bring. From workload containerization to data privacy and governance to hybrid analytics environments, you can expect to see a number of noteworthy advances in the Big Data space.
Rapid Growth of Microservices and Kubernetes Will Transform BI and Analytics
One of today’s biggest megatrends is the rise of microservices and Kubernetes. Together, these technologies take what used to be monolithic and disperse it, essentially enabling a new way to scale workloads. Just like scaling the infrastructure before it, scaling workloads will have a dramatic effect on innovation. In 2019 enterprise architects will view microservices and container orchestration as critical architectural components of BI and analytics platforms. Businesses will focus on more advanced access to big data to feed predictive and advanced analytics models and create automated insights. Big data will also drive new initiatives to enable more flexible delivery of analytics and support for improved data management. At the same time, modern IT architectures powered by cloud, containers and microservices will also create new challenges in data monitoring, management and governance for enterprises.
Increased Governance Will Change How Big Data is Managed
The European Union’s GDPR (General Data Protection Regulation) has completely changed the dynamics of data ownership, privacy and protection. Before GDPR, corporations like Facebook and Google “owned” and controlled the user’s data and could do with it almost anything they wanted. GDPR, which affects any organization collecting data from EU subjects and conducting business in the EU, shifts control to the individual and places strict compliance requirements on what data organizations can collect, as well as how that data can be retained and used. For example, GDPR enables an individual to request that a company delete their personally identifiable information (PII). GDPR also requires companies to anonymize their data and prove the necessity of retaining identifying information. But rising data privacy and governance concerns outside of the EU, combined with a constant string of high-profile data breaches, will eventually result in similar legislation being proposed in the U.S…and change the way we treat big data.
Organizations Will Rethink Data Monetization
Data has become the most valuable resource in the world. Through a unique combination of data collection, warehousing, and analysis, companies are finding new ways to drive their businesses and tackle complex problems. For these companies, data monetization is the act of turning data into revenue. These organizations perceive the term “monetization” as representing a “value in exchange” (what someone is willing to pay me for my data). That will change, as more companies embrace the notion of data monetization as meaning “value in use” (leveraging the insights buried in the data to create new sources of value). Companies that put this new way of thinking into action will fundamentally change their business practices. The transformation will be greatest in their sales and marketing departments, as they leverage data as a strategic asset and adopt analytical practices in search of a competitive edge.
More Businesses Will Adopt Data-driven Decision-making
Advances in machine learning and big data analytics are enabling us to make predictions about future trends by analyzing patterns, and thus can improve outcomes associated with the decision-making process. Many organizations still rely heavily on intuition or “gut instinct” to make important business decisions. While gut feel can sometimes be effective, it is imperfect. Highly successful companies like Amazon rely heavily on hard data to inform fact-based decision-making. 2019 will be a tipping point as more organizations shift their cultures to become more data-focused when making strategic and tactical decisions and develop greater commitment to the data value chain for analytics purposes. Organizations that adapt their business models, strategies and processes to become more data-driven will thrive, while laggards will be disrupted and fall by the wayside.
Hybrid Data Analytics Environments Will Be the Norm
Organizations are challenged with standardizing on a single big data environment as technologies advance and the economics of cloud vs. on-prem continue to change. While the trend over the past few years has been to embrace the cloud for big data analytics with machine learning, some companies are retaining or moving back to on-premises because of the economics of cloud. Even with the additional automation, cloud can be significantly more expensive than on-premises, especially for non-dynamic workloads. We see some companies implementing both on-premises and cloud-based Hadoop and Spark infrastructures (sometimes using multiple cloud vendors). Organizations have many choices, and that means data ecosystems will continue to be a complex arrangement of different environments for the foreseeable future.
