Managing multi-tenant big data clusters is complex. Pepperdata partners with you to deliver predictable performance, empowered users, managed cost, and managed growth with proven big data APM.
Pepperdata delivers proven big data APM products, operational experience, and deep expertise.
PHILIPS WELLCENTIVE
“Before Pepperdata, we experimented with various approaches to solve our performance issues, but we couldn’t see deep enough into the cluster. Pepperdata Platform Spotlight shined a bright light into our Hadoop environment and provided the detailed data that helped us isolate and resolve the problem.”
RUBICON PROJECT
“At Rubicon Project, having the appropriate visibility and insight into our Big Data applications is extremely important when delivering detailed reports to our clients and meeting our SLA. We challenged Pepperdata to find a solution to profile our applications before going to production that would help us maintain our SLA to our customers as we introduce new applications. Pepperdata listened to us and quickly understood the problem we were trying to address.”
CHARTBOOST
“Chartboost is the world’s largest mobile games-only advertising platform, reaching one billion active players around the world every month. Chartboost utilizes Apache Spark on large Amazon EC2 Hadoop clusters for machine learning and ETL workflows. Understanding Spark application performance in these complex environments is always a challenge. As a current user of Pepperdata Platform Spotlight, it has been great to work with Pepperdata on the development of the Application Spotlight self-service portal software. It will give us a comprehensive insight into Spark jobs.”
EXPEDIA
“The level of support and expertise that we receive from the Pepperdata team made a big difference to us. Pepperdata worked closely with us on our Cluster Analyzer implementation to ensure success. With Pepperdata Capacity Optimizer, the DevOps team runs more jobs, faster. We’ve seen a big performance boost across the cluster and have a much more efficient data footprint. Using the Pepperdata dashboard to see application-level metrics, unique custom views, resource utilization per workload drill-downs, and hardware utilization by various workgroups has significantly improved the way that we manage and troubleshoot.”
See who’s using Pepperdata big data APM solutions to achieve predictable performance, empower users, and to manage cost and growth for their big data investment.
Evaluating and purchasing a big data performance solutions can be complicated. We’ve made it easy to make sure you don’t miss anything with our Pepperdata Big Data APM Solutions Overview.
See firsthand how Pepperdata solutions bring APM to every phase of the big data DevOps cycle with solutions for monitoring, tuning, troubleshooting, and improving collaboration between teams. Request a trial today!
Extensive Enterprise Reporting Capabilities for Application Spotlight and Platform Spotlight and Expanded Professional Services Unveiled at Strata Data NYC
NEW YORK — September 11, 2018 (Strata Data New York 2018, Booth 741) — Pepperdata, the leader in Application Performance Management (APM) for big data success, announced enterprise-grade features to its APM suite that include auto-tuning, enhanced recommendations, and management and operational reporting, powered by an easy-to-use self-service interface. The company also announced professional services offerings that include best-practices, performance planning, capacity planning, and architecture design for big data success.
The company’s new professional services are directly enabled by the vast amount of metrics — 600 trillion data points every year — that Pepperdata collects from tens of thousands of nodes every few seconds. This data provides unique insight into all aspects of operationalizing big data applications. Pepperdata is unique in its ability to deliver not only enterprise-grade software, but also expertise, experience and knowledge that ensures big data success.
“Customers are demanding more than features and function from us — they’re asking us to become partners in making sure their big data investments yield business results,” said Ashfaq Munshi, Pepperdata CEO. “We are the only company offering expert services along with a solution delivering instantaneous time-series data that provides precise insight relevant to enterprise platforms and applications.”
Proven Products
The Pepperdata APM suite — comprised of Platform Spotlight and Application Spotlight — enables tight collaboration between developers and operators, improves overall efficiency and performance, and enables enterprises to do more with their existing big data investments.
Platform Spotlight provides infrastructure and capacity managers with:
Application Spotlight provides developers with:
Operational Experience and Deep Expertise
Pepperdata continuously monitors over 250 production clusters across its customer base — over 30,000 nodes across all Big Data distributions and hardware configurations — for a total 550 million jobs and 600 trillion data points every year. Coupled with its success serving Fortune 100 customers, this uniquely broad set of data empowers Pepperdata to help customers:
Helpful Links
About Pepperdata
Pepperdata is the leader in Application Performance Management solutions and services for big data success, solving application and platform issues throughout the stack for developers as well as capacity and infrastructure managers. The company partners with its customers to provide proven products, operational experience, and deep expertise to deliver predictable performance, empowered users, managed costs and managed growth for their big data investments, both on-premise and in the cloud. Leading companies like Comcast, Philips Wellcentive and NBC Universal depend on Pepperdata to deliver big data success. Founded in 2012 and headquartered in Cupertino, California, Pepperdata has attracted executive and engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata investors include Costanoa Ventures, Signia Venture Partners, Silicon Valley Data Capital and Wing Venture Capital, along with leading high-profile individual investors. For more information, visit www.pepperdata.com.
###
Contact:
Samantha Leggat
samantha@pepperdata.com
Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.
Pepperdata Anticipates Doubling Team Within a Year to Support Increase in Seven-Figure Sales Deals
CUPERTINO, Calif. — September 4, 2018 — Pepperdata, the leader in Application Performance Management (APM) for big data success, announced the appointment of two executives reporting to CEO Ash Munshi. The appointments include Charles Marker as Vice President of Engineering and Dan Marx as Vice President of Sales. This announcement comes on the heels of the company’s latest funding, which Pepperdata will allocate to hiring and product development as it continues to deliver on feature requests from customers to support their mission-critical big data deployments.
Charles Marker joins Pepperdata as VP of Engineering from his previous position as Global Head of Engineering at Guidewire Software. Prior to Guidewire, Mr. Marker held Engineering VP positions at Kontagent, Yahoo, Qualcomm and Atheros. Dan Marx, who has been with Pepperdata since 2014, has been named VP of Sales. Mr. Marx brings deep expertise and experience in enterprise sales, including extensive success in big data technology sales at WANdisco and Zettaset.
“We are excited about the contagious enthusiasm and deep expertise Charles and Dan bring to Pepperdata,” said Mr. Munshi, Pepperdata CEO. “As we close more and more seven-figure deals, we are pleased to have the funding necessary to make appointments like these, and we will continue expanding to support the tremendous growth we’re experiencing. We anticipate more than doubling our team within a year.”
“Pepperdata is the leader in Application Performance Management for big data, delivering scalable solutions that enable Fortune 100 companies to achieve successful outcomes from their investments. We continue to be impressed with Pepperdata’s ability to facilitate adoption by these leading companies by identifying use cases that benefit from APM. We are pleased to work with them as they continue to execute their strategy,” said Jim McLean, Managing Director at Silicon Valley Data Capital.
“We were impressed to see the world’s biggest and best AI-driven companies already using Pepperdata so their Hadoop and Spark clusters perform at scale. Pepperdata helps ensure these global brands in e-commerce, voice applications and consumer banking optimize both the productivity and performance of their big data practices. Their continued team and company growth is exciting,” said Greg Sands, Managing Partner at Costanoa Ventures.
Since its founding in 2012, Pepperdata has established itself as a leader in APM for big data success, delivering proven products, operational experience, and deep expertise for its customers. Pepperdata is deployed at Fortune 100 companies in financial services, retail, healthcare, telecommunications and more, totaling more than 250 production clusters with 30,000 nodes spanning all big data distributions and hardware configurations. With the level of data the company collects — over 550 million jobs and 600 trillion data points annually — and its extensive global enterprise experience, Pepperdata is the wise choice for companies looking to get more value and optimal performance from their big data investments.
Pepperdata will be exhibiting at the Strata Data Conference at the Jacob Javits Center (booth 741) in New York City, September 12th and 13th.
Helpful Links
About Pepperdata
Pepperdata is the leader in Application Performance Management solutions and services for big data success, solving application and platform issues throughout the stack for developers as well as capacity and infrastructure managers. The company partners with its customers to provide proven products, operational experience, and deep expertise to deliver predictable performance, empowered users, managed costs and managed growth for their big data investments, both on-premise and in the cloud. Leading companies like Comcast, Philips Wellcentive and NBC Universal depend on Pepperdata to deliver big data success.
Founded in 2012 and headquartered in Cupertino, California, Pepperdata has attracted executive and engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata investors include Costanoa Ventures, Signia Venture Partners, Silicon Valley Data Capital and Wing Venture Capital, along with leading high-profile individual investors. For more information, visit www.pepperdata.com.
###
Contact:
Samantha Leggat
Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.
CUPERTINO, Calif. — March 1, 2018 — Pepperdata, the Big Data Performance company that provides developer and operator solutions to optimize application and cluster performance in Big Data environments, announces Application Spotlight. This self-service portal enables Big Data application developers to generate application-specific recommendations to improve application performance, highlight applications that need attention, automatically identify bottlenecks, and alert on duration, failure conditions, and resource usage.
Application Spotlight helps developers to search for all applications running on the cluster or just the specific applications that they are interested in, compare current and previous runs, and visualize Spark applications and its stages for easy root cause failure analysis and performance tuning.
“Application Spotlight provides relevant application information, insights, and calls to action, all in one place, so that developers can easily and quickly perform these tasks,” said Ash Munshi, CEO of Pepperdata. “In addition to helping developers make jobs go faster, Application Spotlight enables them to be better tenants in multi-tenant clusters by showing them how to write optimal jobs and more efficiently use their queue and cluster resources with practical, innovative application performance management solutions. Application Spotlight enables developers to quickly understand performance impacts and get recommendations on how to better optimize their jobs.”
Big Data Performance Solutions for Both Developers and Operators
The Pepperdata product suite provides solutions for monitoring, tuning, troubleshooting, applications and clusters along with automated cluster optimization. Pepperdata allows enterprises to:
Pepperdata products improve collaboration between development and operations teams by providing both an application as well as cluster view of performance. Pepperdata products, are used to monitor and manage mixed workloads from frameworks such as Spark, MapReduce, Kafka, Tez, Solr, and Impala.
See a Demo of Application Spotlight at Strata San Jose
Pepperdata is sponsoring, speaking, and exhibiting at Strata Strata Data Conference on March 6–8 at the San Jose Convention Center. Please visit Pepperdata Booth to discuss your requirements and see a demonstration of Pepperdata Application Spotlight. For information about our Strata schedule, go to www.pepperdata/events.
Helpful Links
About Pepperdata
Pepperdata is the big data performance company. Leading companies such as Comcast, Philips Wellcentive, and NBC Universal depend on Pepperdata to manage and improve the performance of Hadoop and Spark. Enterprise customers use Pepperdata products and services to troubleshoot performance problems in production, increase cluster utilization, and enforce policies to support multi-tenancy. Pepperdata products and services work with customer Big Data systems both on-premise and in the cloud.
Founded in 2012, Pepperdata has raised $20M from investors including Citi Ventures, Signia Venture Partners and Wing Venture Capital, and attracted senior engineering talent from Yahoo, Google, Microsoft and Netflix. Pepperdata is headquartered in Cupertino, California. For more information, visit pepperdata.com.
###
Pepperdata and the Pepperdata logo are registered trademarks of Pepperdata, Inc. Other names may be trademarks of their respective owners.
Media Contact
Jim Dvorak
Offleash for Pepperdata
(415) 735-1622
pepperdata@offleashpr.com
Today’s blog discusses what it means to be an operations manager in a Hadoop environment and covers the Pepperdata Platform Spotlight, our operations-focused solution. There are big obstacles to big data access, not the least of which is the fact that you’re dealing with a complex multi-tenant distributed system.
When dealing with application and platform performance in a multi-tenant environment, you need to understand how to properly tune both — by right-sizing containers and effectively allocating resources appropriately. There are countless questions to answer:
If you have just a few users, it’s probably not an issue, but when users number in the hundreds or even thousands, it will be a real challenge to ensure they are satisfied.
APM for Big Data Success
To ensure big data platform success, you need operational performance truth across the entire cluster. You need to understand what your system is doing, how it’s being affected by users, how your applications are performing, and what resource changes need to be made to optimize it for cost effective utilization of your resources.
If you don’t have these insights, it will be impossible to accurately and successfully scale your platform. For instance, if you know you’re only using 40% of your resources, why would you plan to add more hardware? You also want to receive alerts on important events and get recommendations to change configurations. For these things, you need deep knowledge experience on everything having to do with the big data platform.
Pepperdata Differences
Pepperdata Platform Spotlight addresses all the questions you face as an operations manager. It helps you determine why you are seeing bottlenecks, visualizes usage trends, and identifies optimal configuration settings to make users more productive.
360°Platform View
First, Pepperdata provides you with what we call a 360° platform view. This is your single source of operational and performance truth all the relevant real-time cluster operations data. With complete instrumentation of your entire big data ecosystem, Platform Spotlight tells you how much memory applications are using, identifies wasteful applications, and quantifies how applications are impacting your platform. It helps you ensure developers have what they need when they need it.
APM Operations Manager 360° Platform View
Platform Tuning
Pepperdata capacity optimization can help you increase throughput by leveraging AI-driven resource management to automatically tune cluster resources and recapture wasted capacity, reduce hardware costs, and rightsize resource allocation based on actual real-time capacity.Platform tuning can also help you determine which applications and users are reading and writing small files, thereby impacting the overall performance of your HDFS environment.
Platform Recommendations
You’re challenged with configuring and sizing critical resources running on multi-tenant clusters with mixed workloads. Pepperdata helps you achieve optimal application and cluster performance on multi-tenant systems with recommendations to right-size containers, queues and other resources.
Platform Alerting
You receive so many alerts that aren’t specific enough to isolate and resolve problems. Pepperdata exposes all the right granular data, allowing you to create alerts that quickly pinpoint root causes, identify rogue users and applications, and identify network errors, disk latency and potential capacity shortages.
360° Reporting
Capacity and resource planning and management is difficult without the right data and insights. With all of the necessary data about your cluster (data about hosts, queues, users, applications, etc.), Pepperdata enables you to visualize and predict the growth of the cluster and make an informed decisions around capacity and resource requirements and recapture resources.
Pepperdata Platform Spotlight provides you with a single source of truth to quickly diagnose performance issues anywhere in the environment and make resource decisions based on user priorities and needs. In addition to a comprehensive view of your big data platform, Pepperdata offers proven experience on hundreds of clusters at Fortune 1000 companies. As a trusted advisor helping enterprises establish and follow best practices, Pepperdata can provide you with guidance on the best architecture using real-world experience derived from some of the world’s biggest clusters.
Contact sales@pepperdata.com today and learn how we can help you.
More info:
As you know by now, Cloudera and Hortonworks have announced their plans to merge. The companies reportedly have a combined fully-diluted equity value of $5.2 billion and expect to generate approximately $720 million in revenue.
As after any notable merger, articles and commentaries surface, asking things like whether this represents the death of Hadoop and if the consolidation was a move for the two companies to improve their margins. Those will get played out eventually but they’re fundamentally dependent on whether the companies can execute the merger successfully and minimize the disruption to customers.
Having gone through several mergers of this size and larger, I have experience with the challenges involved. In this blog, I’m going to focus on the challenges with the merger and what it represents for the competition, the companies themselves, and most importantly, their customers.
Leadership Team
Although it appears that Cloudera is slightly more dominant with the 60/40 split given the way the companies are structuring the deal, it’s likely to be more of the merger of near equals. In my experience, this operating paradigm is harder to pull off than a merger that involves a dominant company, where that dominant culture enforces itself and execution risk is minimized with the dominant management team remaining in place.
With a merger of near equals, everything needs to be carefully negotiated, including product line, marketing strategy, pricing and positioning, which can lead to disagreements and politics that take time to work out. The first thing the companies need to determine is what the executive team will look like and ensure that the leaders work well together de-facto – not just de-jure. That executive team composition will help identify the culture of the new company and how it will go about making decisions — everything from sales and marketing and positioning to product and customer support decisions.
Products
In addition to management and culture challenges, with this merger there’s a huge overlap of product lines, which creates issues to be resolved. Important decisions regarding which product to continue and which to phase out are fraught with politics. Again, these discussions and decisions take time, which will undoubtedly cause some confusion and uncertainty in the marketplace. Customers will need to be told sooner than later what products survive and how they will transition from one to another. This will be a great opportunity for MapR and others to take advantage of the uncertainty.
Pricing and Vendor lock-in
Pricing is going to be an interesting issue for customers using products from both Cloudera and Hortonworks. Consolidation between the two largest vendors creates lock-in risk for some enterprises in that they become too dependent on a single vendor and the vendor could exert pricing leverage. This will force the largest customers to evaluate alternatives simply to mitigate risk. Additionally, if the spend is large enough with the combined entity, tech savvy enterprises will need to consider going to Apache and supporting the infrastructure themselves as it is likely cheaper in the near term and puts the control in their own hands.
The Cloud and AI
There are two dominant use cases for Big Data: ETL and Analytics. In the case of Analytics, a direction on which the new company will undoubtedly focus, the cloud is a big competitive concern during the consolidation process. Without a well-defined and articulated cloud strategy, Cloudera and Hortonworks face the risk that vendors like Microsoft and Amazon or Databricks and Snowflake could come in and displace them.
AI is also a long term strategic goal for the new company but it is hard to see how they can even get to par with Google, Microsoft or Amazon. The big guys are far more knowledgeable and if the data gets moved to cloud object stores negating one of the key principles of Hadoop – cheap storage, it will be increasingly more difficult to get and retain customers.
Employees
As the merger moves along, workforce cuts are inevitable. There will be pressure for retention because the first people to leave are usually those you don’t want to lose because it’s easy for them to find other jobs. With companies and talent firms — including those at the cloud and technology vendors I’ve already mentioned — surely using this opportunity to recruit, we’ll start to see key people leaving, particularly in engineering.
From a customer perspective, these losses will raise significant questions about the stability of the new company. It may also cause some customers to question whether Hadoop is in fact dead and should they consider accelerating their move to the cloud or another solution.
What It Means for Pepperdata
With a merger like this, discussions inevitably arise about the future of the companies as well as the technology. The management team at the newly formed company will figure things out. However, that does not mean it is without uncertainty for the next 12-18 months and they’re going to have to plan and manage that transition very carefully.
At Pepperdata, we’re rooting for the company’s success. Regardless of the outcome, we have customers relying on us to ensure their big data platforms and analytics solutions are running well. We play an important role and it will get even more crucial going forward.
Pepperdata is not married to Hadoop or YARN or any one commercial solution. With proven products, experience and expertise and our demonstrated ability to work with Kubernetes, we are focusing on workloads that go beyond Hadoop and HDFS. We will continue on that path irrespective of this merger. As a matter of fact, it will likely accelerate our progress.
As a capacity manager, you’re responsible for managing costs and growth as well as driving business by helping developers succeed with their applications. Both the applications and the platform must to be high-performing, and developers must understand the aspects of the platform that affect their applications. It’s one thing to get a platform up and running and allow users to run applications; it’s another thing to make truly effective decisions and changes.
Questions
Capacity managers are in a very unique position because they’ve got a responsibility to the business to ensure that users are satisfied and the business is benefiting from optimal performance and capacity. With your APM solution, you should be able to answer yes to the following questions:
Answers
To answer these questions and leverage your existing investment, you need an APM solution that:
A comprehensive APM solution collects all relevant data, metrics on both application workloads – user behaviors, errors, response times, API calls, etc.– and the environment–resource utilization, data sources, etc.–to obtain accurate and useful insights. By measuring and tracking user transactions, you can understand how applications behave and whether SLAs are being met. Environment measurements help you identify patterns in resource usage and capacity demands. The goal of measuring and analyzing these is to deliver an excellent user experience and get the most out of your infrastructure.
Pepperdata: Massive Scale APM
Pepperdata provides a complete view of your entire cluster so that you can uncover performance problems and identify patterns that impact the entire application environment and make intelligent resource decisions. Pepperdata continuously collects extensive data on hundreds of real-time metrics from all of your applications and infrastructure resources — metrics about CPU, RAM, disk I/O, and network usage for every job, task, user, host, workflow, and queue. This data is not available with any other tool or solution. Pepperdata helps you understand and improve platform performance, reduce mean time to problem resolution and increase capacity utilization by 30-50% without adding new hardware. In addition to surfacing performance bottlenecks, Pepperdata provides automatic tuning for recurring applications, delivers app-specific recommendations and allows you to set up alerts on specific behaviors and outcomes to avoid the risk of failure.
360° Platform View
You need a holistic source of operational and performance truth across your clusters. Pepperdata allows you to access real-time and historical data about the cluster, including system demand, abusive users, and wasteful applications, queues, and container sizes. Drill down or zoom out to analyze any application and understand its performance in the context of the entire multi-tenant cluster. With complete instrumentation of your Big Data ecosystem, Pepperdata allows you to monitor any process (Kafka, Impala, HBase, Hive, MapReduce, Tez, Spark, IBM BigSQL, LLAP) to help you quantify the impact that applications have on the cluster as well as the impact that the cluster has on applications.
Platform Tuning
Pepperdata tracks actual hardware resource usage of every node on the cluster in real time, helping to improve performance and efficiency of your cluster by maximizing existing resources, auto-scaling cluster resources for peak efficiency and rightsizing resource allocation based on real-time capacity. This allows more tasks to be run on nodes that have available free resources at the moment, increasing capacity without adding hardware.
Platform Recommendations
Pepperdata generates targeted recommendations to improve application performance, highlights applications that require attention, automatically identifies bottlenecks, raises issues around duration, failure conditions, and resource usage. In addition, Pepperdata helps you accurately forecast resource needs by identifying growth trends across groups and users.
Platform Alerting
Reduce time to problem resolution using comprehensive and detailed performance data. Leveraging its vast collection of real-time hardware metrics, Pepperdata allows you to create custom alerts and queries and filter and compare by different dimensions to quickly identify what resource or app is causing a problem and which user submitted it.
Insights through Reports
Pepperdata generates comprehensive planning reports containing trends and patterns based on real-time and historical data about the cluster so that you can plan capacity for predictable performance. Use these reports to accurately plan for growth, identify which users and applications are wasting resources, and attribute accurate costs to specific users and business units.
The Bottom Line
Pepperdata Platform Spotlight provides you with a single source of truth so that you can quickly diagnose performance issues anywhere in the environment and make resource decisions based on user priorities and needs. In addition to a comprehensive view of your big data platform, Pepperdata offers proven experience on hundreds of clusters at Fortune 1000 companies. As a trusted advisor helping enterprises establish and follow best practices, Pepperdata can provide you with guidance on the best architecture using real-world experience derived from some of the world’s biggest clusters. Tap into our experience and expertise to achieve more uptime, better performance, improved capacity planning, faster case resolution, and proactive issue avoidance and prevention.
Contact sales@pepperdata.com today and learn how we can help you.
More info:
Request a trial to learn more about how Pepperdata can help you optimize application performance.
