It’s a new year, and time to turn our attention to what the future may bring. From workload containerization to data privacy and governance to hybrid analytics environments, you can expect to see a number of noteworthy advances in the Big Data space.
Rapid Growth of Microservices and Kubernetes Will Transform BI and Analytics
One of today’s biggest megatrends is the rise of microservices and Kubernetes. Together, these technologies take what used to be monolithic and disperse it, essentially enabling a new way to scale workloads. Just like scaling the infrastructure before it, scaling workloads will have a dramatic effect on innovation. In 2019 enterprise architects will view microservices and container orchestration as critical architectural components of BI and analytics platforms. Businesses will focus on more advanced access to big data to feed predictive and advanced analytics models and create automated insights. Big data will also drive new initiatives to enable more flexible delivery of analytics and support for improved data management. At the same time, modern IT architectures powered by cloud, containers and microservices will also create new challenges in data monitoring, management and governance for enterprises.
Increased Governance Will Change How Big Data is Managed
The European Union’s GDPR (General Data Protection Regulation) has completely changed the dynamics of data ownership, privacy and protection. Before GDPR, corporations like Facebook and Google “owned” and controlled the user’s data and could do with it almost anything they wanted. GDPR, which affects any organization collecting data from EU subjects and conducting business in the EU, shifts control to the individual and places strict compliance requirements on what data organizations can collect, as well as how that data can be retained and used. For example, GDPR enables an individual to request that a company delete their personally identifiable information (PII). GDPR also requires companies to anonymize their data and prove the necessity of retaining identifying information. But rising data privacy and governance concerns outside of the EU, combined with a constant string of high-profile data breaches, will eventually result in similar legislation being proposed in the U.S…and change the way we treat big data.
Organizations Will Rethink Data Monetization
Data has become the most valuable resource in the world. Through a unique combination of data collection, warehousing, and analysis, companies are finding new ways to drive their businesses and tackle complex problems. For these companies, data monetization is the act of turning data into revenue. These organizations perceive the term “monetization” as representing a “value in exchange” (what someone is willing to pay me for my data). That will change, as more companies embrace the notion of data monetization as meaning “value in use” (leveraging the insights buried in the data to create new sources of value). Companies that put this new way of thinking into action will fundamentally change their business practices. The transformation will be greatest in their sales and marketing departments, as they leverage data as a strategic asset and adopt analytical practices in search of a competitive edge.
More Businesses Will Adopt Data-driven Decision-making
Advances in machine learning and big data analytics are enabling us to make predictions about future trends by analyzing patterns, and thus can improve outcomes associated with the decision-making process. Many organizations still rely heavily on intuition or “gut instinct” to make important business decisions. While gut feel can sometimes be effective, it is imperfect. Highly successful companies like Amazon rely heavily on hard data to inform fact-based decision-making. 2019 will be a tipping point as more organizations shift their cultures to become more data-focused when making strategic and tactical decisions and develop greater commitment to the data value chain for analytics purposes. Organizations that adapt their business models, strategies and processes to become more data-driven will thrive, while laggards will be disrupted and fall by the wayside.
Hybrid Data Analytics Environments Will Be the Norm
Organizations are challenged with standardizing on a single big data environment as technologies advance and the economics of cloud vs. on-prem continue to change. While the trend over the past few years has been to embrace the cloud for big data analytics with machine learning, some companies are retaining or moving back to on-premises because of the economics of cloud. Even with the additional automation, cloud can be significantly more expensive than on-premises, especially for non-dynamic workloads. We see some companies implementing both on-premises and cloud-based Hadoop and Spark infrastructures (sometimes using multiple cloud vendors). Organizations have many choices, and that means data ecosystems will continue to be a complex arrangement of different environments for the foreseeable future.