If you haven’t heard much about Big Data, that’s about to change. Big Data is going viral.
If you have heard of Big Data, odds are you only heard the term for the first time in the last few months. In January 2011, there were only 20,000 Google searches on the term “Big Data,” but since then, Big Data has gained steam. Last month, the number of searches exceeded 220,000, more than a tenfold increase.
Source: Google Insights. http://www.google.com/insights/search/#q=%22big%20Data%22&date=1%2F2008%2049m&cmpt=q
After spending the morning at the Mass Technology Leadership Council’s Big Data Summit we’re convinced that Big Data is for real. There will be a lot of great Big Data businesses built over the next few year, but also a lot of failed efforts.
So where might the Big Data opportunities for your business be…and how do you avoid the pitfalls?
To answer these questions, we'll start with definition of Big Data: Big Data is a catch-all term for everything related to really, really big data sets.
Social networking, web surfing, electronic health records, RFID chips, cars that track their own locations, and many, many more sources are creating massive amounts of data (what websites you’ve visited, where your car has been, what medical tests you’ve had, why you had them, and what the results were, etc.). In Big Data Land, forget Terabytes. You’ve now entered the world of hundreds of Terabytes, Petabytes (1,000 Terabytes) and soon, Exabytes (1 million Terabytes).
While Big Data can get very complicated, it can generally be divided into 3 areas.
1. Big Data Infrastructure
The Opportunity: As data sets get larger and larger, there will need to be a new generation of infrastructure products just to keep up with the volume. These include faster storage, new database architectures, and parallel processing tools.
The Pitfall: There are many different types of Big Data data sets and many different kinds of processing demands. Some infrastructure items, like Fast Storage, will have very broad market appeal. But others may have a very limited market. You might be able to build the fastest system for calculating arrays on Petabyte data sets, but if there are only 5 companies who need it, it isn’t a solution you can build a company on.
The Opportunity: Data turns into value when it can be applied to solving real world problems. New types of data and new algorithms will enable companies to develop compelling new solutions that will bring huge returns. Solutions like an automated Physician’s Assistant that helps doctors more accurately diagnose patients, a navigation system that automatically adjusts for traffic, and a location aware mobile ad platform that serves you ads for things you really want are all examples of Big Data analytics in action.
The Pitfall: Many efforts to mine and utilize Big Data will fall short – because the data isn’t good enough, the algorithm isn’t robust enough or the implementation of the results is too difficult. Pandora (www.pandora.com) is a great example of a Big Data-driven music recommendation service that delivers real value. However, for every Pandora, there were 20 other recommendation services that fell flat.
The Opportunity: There is an incredible amount of valuable data being created every day that is just waiting to be turned into a product or service.
The Pitfall: Not every data set can be spun into gold. Frequently, we hear companies talk about monetizing their data. However, only rarely do the revenue and profits materialize. Unless you gather data with a clear understanding of how it will be used and what the value proposition is, it’s unlikely that what you have will be sufficient to turn into a valuable product.
Looking to get into Big Data? We can help you find the opportunities and avoid the pitfalls. Contact us at firstname.lastname@example.org
If you want to learn more about Big Data, here are two great reports:
Big Data and Analytics: A Major Market Opportunity by the Mass Technology Leadership Council
Big Data: The Next Frontier for Innovation, Competition and Productivity by the McKinsey Global Institute