Ninety percent of the data in the world today has been created only in the last two years, according to IBM. With the increase of mobile devices, social media networks, and the sharing of digital photos and videos, we are continuing to grow the world’s data at an astounding pace. Data is big…and getting bigger.
Big Data is a big thing. It is not a passing fad. It has become an all-encompassing, somewhat sprawling term that has defied conventional definition. Seemingly, it has as many definitions as it does applications. In fact, it is most accurately described by its dimensions – the so-called “5Vs” - Volume, Velocity, Variety, Veracity and Value. Solutions for harnessing and leveraging Big Data have also been elusive. What is clear, however, is that technology alone is not the answer.
Data has always been used to develop high-level metrics and business intelligence. Smart organizations have long relied on data to help make strategic business decisions. But the power and allure of Big Data is how it enables organizations to leverage unconventional data points: the information that was previously ignored because there was no reasonable way to process it.
The key question is, “How do we extract big knowledge from big data?”
Unprecedented access to information (according to former Google CEO Eric Schmidt, every two days now we create as much information as we did from the dawn of civilization up until 2003) and emerging technology (allowing us to harness different types of data – both structured and unstructured) have resulted in the rise of big data analytics, which hold the promise of helping companies make more informed business decisions by enabling data scientists, predictive modelers and other analytics professionals to analyze large volumes of transaction data, as well as other forms of data that may be untapped by conventional business intelligence (BI) programs. This could include photos, sensor data, video or voice recordings, web server logs, Internet clickstream data, social media content and social network activity reports.
The technical challenge of using big data to drive innovation and business growth is only part of the solution. A pervasive culture of change management needs to be in place to reap the full benefits of the effort.
Data Democracy. One of the most critical aspects of big data is how it can flatten hierarchical decision making. Data is power and it is the great equalizer. Most organizations have operated in an environment where there is a paucity of data. This has resulted in a certain “fill in the blank mentality,” and decisions are made at the highest pay grade, on the basis of experience senior leadership has built up and patterns and relationships they’ve observed and internalized. This is a flawed approach. Data should be the final arbiter, not executive fiat. Senior leadership needs to encourage data-driven decision making by everyone in the organization.
The War for Talent. The new breed of analytics specialists need to have a combination of skills including statistical techniques, applied mathematical methods, advanced machine learning algorithms, data visualization, and business and communications skills. Many of the key techniques for using big data are rarely taught in traditional university courses. Perhaps even more important are skills in cleaning and organizing large data sets; the new kinds of data rarely come in structured formats. Not surprisingly, people with these skills are hard to find and in great demand. Human Resource departments need to develop new strategies and approaches to acquiring and retaining talent.
Ever Vigilant. According to Forrester, firms use only five percent of the data available to them, while created data is growing at 40 percent to 50 percent annually and only 25 percent to 30 percent of that total is being captured. This places a premium on constantly monitoring the technological landscape for tools that address the 5Vs of big data. Improved predictive analytics, real-time computational capabilities and modelling techniques all dictate that technology upgrades be a significant part of any big data strategy. Movement towards open source software has kept cost of change comparatively low. Organizations need to develop an adaptive data strategy to address emerging technologies.
Big Data brings with it big promise. There are many challenges, but the rewards are clear. As Peter Drucker noted, “You can’t manage what you don’t measure.” The allure of big data is that it provides organizations with an unprecedented capability to measure.
Learn more. Reach out to Scott Leeb.