In a May 2011 special research report, Big data: The next frontier for innovation, competition, and productivity, the management consulting firm McKinsey put forth the case that “Big data will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus.” The McKinsey report went on to note that, “The amount of data in our world has been exploding. Leaders in every sector will have to grapple with the implications of big data, not just a few data-oriented managers. The increasing volume and detail of information captured by enterprises, the rise of multimedia, social media, and the Internet of Things will fuel exponential growth in data for the foreseeable future.”

General usage of the term “Big Data” can be traced to the McKinsey report and similar reports from IBM that ensued around this time. The McKinsey report was prescient in its observations that “Leaders in every sector will have to grapple with the implications of big data, not just a few data-oriented managers.” In retrospect, this was the key insight. From this point forward, interest in data would no longer be limited to the purview of “a few data-oriented managers,” but rather would become the purview of “leaders in every sector.” The McKinsey report went on to describe the advent of the era of Big Data as heralding “new waves of productivity growth, innovation, and consumer surplus.” The report contained one important caveat however, noting that these advances were all predicated “as long as the right policies and enablers are in place.”

Continue reading… “The ‘failure’ of big data”