Big data remains a game for the 1 percent. Or the 15 percent, as new O’Reilly survey data suggests. According to the survey, most enterprises (85 percent) still haven’t cracked the code on AI and machine learning. A mere 15 percent “sophisticated” enterprises have been running models in production for more than five years. Importantly, these same companies tend to give more time and attention to critical areas like model bias and data privacy, whereas comparative newbies are still trying to find the On button.
Unfortunately, for those companies hoping to close the data science gap with automated shortcuts like Google’s AutoML or through paid consultants, the answer seems to be that getting data science right takes time. There are no shortcuts.
Smart companies focus on the deep end of data
First, it’s important to note that O’Reilly’s survey data comes from a self-selected bunch: people who have attended O’Reilly events or otherwise engaged with the company through webinars or other means. Such people have a proactive interest in data science, even if (as the survey data shows) most aren’t really doing much with it. For those steeped in big data experience, however, this is a great demographic, with those dubbed “sophisticated” having models running in production for more than five years.
One interesting data point that emerges from the survey is how these people talk about themselves. Companies with extensive data experience call a data science spade a data science spade. Those stuck in 1990s “data mining” mindsets prefer “analyst,” as the figure shows.
