Zeotap is one of the world’s leading customer intelligence solutions. They help businesses identify, understand, and target their customers through first-party data platforms. To consistently provide clients with powerful and rich insights, Zeotap runs and analyzes large volumes of big data with Spark.
Zeotap had been processing big data with Spark for a while. However, they were running big data processes on Spark in an unoptimized fashion. The company was struggling to meet SLAs, big data performance was abysmal, and costs skyrocketed.
Pepperdata was able to help the company swiftly address these issues, and optimize their big data operations.
Zeotap Before Pepperdata
The problem with running unoptimized Spark jobs is that it often results in slow, laggy performance and overspending. While Spark enabled Zeotap to work with large quantities of data, the big data on Spark architecture consumed more resources than needed. Spark queries took longer to process as well.
Overspending is a common issue for companies that embrace the cloud and other emerging digital technologies. In one of our studies, we found that 80% of businesses are overshooting their cloud budgets. In another, we discovered that 1 in 3 enterprises will likely spend 20% to 40% more than their initial cloud allocation.
In addition to Zeotap’s overspending problem, the company also struggled with visibility into their big data infrastructure.
Visibility helps enterprises see into their cloud and big data processes in real time. This capability enables them to collect and evaluate real-time performance data from tasks, clusters, and more. The insights they glean helps them generate end-user recommendations for superior performance optimization.
However, Zeotap’s initial data visualization tool, Sparkles, was unable to link data across various clusters and workflows. Monitoring several jobs that go through multiple Google Dataproc clusters wasn’t a breeze either.
Without visibility, Zeotap couldn’t optimize their big data with Spark.
More Customers = More Big Data Problems
As Zeotap’s customer base grew, the scale of their big data issues scaled drastically as well. More customers meant more Spark applications and big data workloads running. As a result, the problems compounded. This is a common issue that we observe with companies.
Without an effective means to track, monitor, and optimize these applications and workloads, Zeotap was at risk of spending beyond its big data budget. Worst, they stood to fail to meet customer expectations and potentially lose business.
The Centrality of Spark
Although Zeotap was struggling to run big data with Spark, abandoning Spark and finding another alternative big data analytics engine was out of the question.Why? Because Spark is now the de facto framework solution for large-scale distributed data processing.
Our big data survey confirms this. Apache Spark is poised to rise to be the most dominant large-scale big data processing platform. Nearly all modern enterprises that rely on cloud and big data technologies use Spark. Moreover, cloud services providers and software vendors utilize Spark to improve their products and support their customers.
These statistics speak volumes to Spark’s centrality:
- 91% of enterprises run big data with Spark because of its impressive performance gains.
- 77% of enterprises say they use Spark because it is easy to use.
- 71% of enterprises prefer Spark because it makes deployment simple.
Zeotap couldn’t replace Spark. But they needed a platform to help them fine tune and optimize their Spark applications and processes.
Big Data with Spark: Optimization via Pepperdata
Turning to Pepperdata, Zeotap found the solution they needed to gain superior visibility into their big data apps and workloads. They now had the power to closely track Spark applications, optimize workloads, and keep costs down to a manageable level.
The result was impressive. In just three months, Zeotap was able to increase the number of tasks performed by nearly 24%. The company also enjoyed cost savings of over $31,000 within the same period.
Want to know more about how Pepperdata helped Zeotap fully maximize their big data with Spark and enjoyed the improved performance and big cost savings? Read the full Zeotap case study now.