Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using traditional data processing applications. The challenges are many and usually include capture, curation, storage, search, sharing, transfer, analysis, and visualization.
Data sets grow in size partly because they are increasingly being gathered by ubiquitous information-sensing mobile devices, aerial sensory technologies (remote sensing), software logs, cameras, microphones, radio-frequency identification readers, and wireless sensor networks.
Advantages
In analyzing the competitive advantage of Big data, a recently released Ivey Business Journal analysis builds a strong case to support the building of large pools of data. The report leading companies can use Big data to outperform their peers.
The real-time and high-frequency nature of this type of data is crucial. For example, “nowcasting” — the ability to estimate metrics such as consumer confidence in real time; something that previously could only be done retrospectively — is becoming more extensively used, which adds considerable power to forecasts.
The era of Big data could yield new management principles as well. In the early days of corporation management, leaders discovered that minimum scale was a key determinant of their competitive success. Likewise, future competitive benefits are likely to build for companies that can not only capture more and better data, but also manage that data effectively.
Disadvantages
Anything “big” should give us pause. Case in point was the economic theory highlighted in Andrew Ross Sorkin’s book and movie of the same name, Too Big to Fail. It asserted that certain financial institutions are so large and so interconnected that their failure would be disastrous to the economy, and therefore they must be helped by government when they experience difficulties.
Big data poses the same ambiguity. Is it really for us? Does our company need it? Do we have the resources to capture Big data successfully? Are costs prohibitive? One may question benefits versus risks endlessly. The more questions you have, the riskier the project will seem.
Assessing the potential negatives suggests another disadvantage of “big.” It frequently permits decision makers to slip into a wait-and-see mode, where very little gets done at all. For example, we’ll wait until our competitors do it, learn from their mistakes, and implement our own project when the time is right. Most often, the time is never right and delays ensue.
Big data in practice
It’s important to distinguish between “lots of data” and Big data. Thomas Sorbo is one of the co-founders of Xeneta, which produces container shipping software and helps businesses get actionable data on their shipping rates and transit.
While he sees traditional technologies as having the ability to analyze large data sets from traditional sources such as warehousing and distribution systems, he feels Big Data takes it to the next level. It “allows companies to harness extremely large data volumes, including non-traditional data types such as text, audio, and video in conjunction with information from business systems in a much more economical fashion, in both batch and in real time modes,” Sorbo notes.
Big Data helps a company achieve its goals and objectives, whether reducing costs or improving efficiency, which is of daily importance for most businesses, or increasing sales, which is of interest to everyone. “When properly used, Big Data will enable management to take a gut-feel decision and quantify it, which lets them take that good idea and make it work,” adds Sorbo.
