The Pepperdata team had a fantastic time at this year’s Strata Data Conference in New York City. All of us are still buzzing from all the discussion of cutting-edge data strategies, techniques, and technologies. 

Here are the top five trends that kept coming up:

1. Big Data Applications are the Future

Everyone at Strata agreed: Big data applications are growing faster and have more impact on the business than ever before. For example, for financial services, the primary big data use case is business analytics that run on Hadoop. Data-driven analytics are key to the current and future competitiveness of financial services companies. By capturing and leveraging massive volumes of data, financial services companies are capitalizing on new data-driven business opportunities.

2. The Cloud Continues to Rise

According to Gartner, this year, the worldwide public cloud service market will reach nearly $222 billion, growing 16% over 2018. Our conversations at Strata confirmed this ongoing rise. Almost everyone we spoke to was either in the cloud, or were actively evaluating cloud options. Most customers said they were intending to maintain their on-premise clusters, but were looking to move many new workloads to the cloud. 

3. Hadoop Is Alive And Well

At Strata, we saw that companies are continuing to grow and expand their Hadoop implementations. There continues to be a strong push towards leveraging open-source standards and technologies, and moving away from proprietary, vendor-controlled technologies. Hadoop is much more than just Hadoop, with YARN, Spark, Kubernetes, and many more open-source technologies comprising its ecosystem. Far from being dead, Hadoop provides a modern data architecture for managing and analyzing data that customers continue to embrace as a foundational technology.  

4. Enterprises Are Struggling With Visibility 

Enterprise companies are struggling to gain visibility into their big data clusters, and struggling to know how they can better utilize their resources. Often, this can be a risk to a company’s overall data strategy. To optimize data operations across the full stack, you must be able to simultaneously manage everything: Hadoop, EMR, HD Insights, Spark, HDFS, S3, and more. On-prem, in the cloud or in hybrid environments. That’s a tall order but very possible today.

5. IT Operations is Seeking Next-Generation Capabilities to Predict and Manage Costs as it Matures Across the Enterprise

IT operations isn’t just tasked with understanding, explaining, and resolving problems in enterprise applications. The next wave of issues is already here: producing financially modeled cost projections and implementing chargeback methodologies which are needed to enhance savings and provide predictable spending across the IT landscape. IT complexity includes financial risk, which requires a more sophisticated approach to supporting multi-tenancy while centralizing IT, and maintaining accountability to the end users of IT services. IT operations is seeking solutions to these new challenges and the race is on to address this area of concern.

Attending the Strata Data Conference was a great experience. There was a variety of people representing many different companies and industries, all of them discussing the future of data in interesting ways. 

We’d love to hear your thoughts on the trends we’ve mentioned here. Did you notice any trends that we may have left out? Want to discuss more about one of the ones mentioned above? Use the hashtag #StrataTrends and tweet us @pepperdata to share your thoughts.


Take a free 15-day trial to see what Big Data success looks like

Pepperdata products provide complete visibility and automation for your big data environment. Get the observability, automated tuning, recommendations, and alerting you need to efficiently and autonomously optimize big data environments at scale.