Blog

Apache Spark™ is a full-fledged data engineering toolkit that enables you to operate on large data sets without worrying about the underlying infrastructure. It achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. While all that may... read more →

Apache Spark™ is a unified analytics engine for large-scale data processing. It is known for running workloads 100x faster than other methods, due to the improved implementation of MapReduce, that focuses on keeping data in memory instead of persisting data on disk.  However, despite its many great benefits, Spark also... read more →

Check out our Introduction to Platform Spotlight webinar, and see how it shows you all the relevant data you need to optimize your cluster performance.     Big Data is difficult. You have all these different tools to help you understand the performance of your clusters, manage and monitor resource capacity, maximize... read more →

2019 was a turbulent and exciting year in the world of Big Data. And here at Pepperdata, we’ve been right in the thick of it. Before we leave 2019 behind, we wanted to look back on the awesome numbers we’ve posted this year.   We couldn’t have put up these... read more →

Gartner recently performed a deep dive into the top trends that will define the world of data through 2020 and beyond. Anyone working in big data – and all Pepperdata users – should be very interested in what they have to say. What does this coming year hold for the... read more →

Hadoop is great. Without it, big data processing would be a whole lot harder. However, manually tuning Hadoop is an inefficient process that reduces job performance by 30-50%. As analytics platforms grow in scale and complexity, both on-prem and in the cloud, managing and maintaining efficiency is a critical challenge.... read more →

Most companies today rely on Hadoop for their big data processing needs. However, such a sophisticated platform can still present pitfalls and challenges that DevOps teams grapple with almost every day. These challenges can usually be summed up into three main issues. Continue reading to find out what these are,... read more →

Everyone in the Big Data world knows Spark. It’s an amazing, powerful piece of software that runs workloads 100x faster than other methods. However, fantastic as Spark is, like all software, it has its challenges. In a recent webinar, we sat down with Alexander Pierce, a Pepperdata Field Engineer, to... read more →

Modern data analytics platforms such as Hadoop and Spark have become central to many Fortune 1000 businesses. As critical components for digital business success, they produce insights that can only be obtained by analyzing the massive amounts of data. The continued growth of these systems can mean that an enterprise... read more →

Big Data LDN returned to London last week. Since launching in 2016, the conference’s overall attendance has tripled to over 8,000 people, making it one of the largest big data events in Europe. Over the two days, attendees heard from over 150 data experts and visionaries on a wide variety... read more →