There is little doubt that Hadoop adoption is growing, and not just among Enterprise-sized organizations, but by small and medium sized businesses as well. In fact, the size of the organization (in terms of revenue and/or employee size which are the two criteria used to measure Enterprise vs. SMB) does not alway correlate to cluster size. Some of the largest Hadoop deployments and users tend to be “small shops”, such as ad tech companies, digital marketing, and analytics departments, that don’t always have the highest numbers of employees.
At Pepperdata, we know that the universe of Hadoop users is growing; every day we encounter a company using Hadoop who we didn’t know previously existed. In an effort to understand this growing market more deeply – such as their key use cases, the size of their Hadoop environments, their key challenges, etc. – we partnered with O’Reilly Media to survey their readership about how and why they are using Hadoop for their business operations. We had 134 readers complete the entire survey and the results were quite interesting.
The survey respondents came from a range of experience, but all are working at companies that are currently running Hadoop in production. The majority of respondents were from software engineering/development, data scientist, or data architect job titles (25%, 17%, and 12% respectively). Almost half (40%) were from the Information Technology industry. The next highest came from Education and Financial Services (11% and 10%). Over 45% have been in production for two years or more, with 15% of those being “advanced users” (4 years or more in production).
Here are some interesting charts to show the size of the Hadoop environments that our survey respondents are directly involved with: