Zettabytes, exabytes, Petabytes – what is up with the big numbers? [Tweet this]
A common approach to discussions on big data is to throw big (and meaningless) numbers at the audience.
How many email messages were sent in the last second – cue BIG NUMBER!
How much data has landed on the Internet in the last two years? Cue EVEN BIGGER NUMBER!
Why should you be thinking about big data? Add up all the big numbers the approach seem to say.
It reminds me of a old joke.
During the US occupation of Iraq the President attends his daily Iraq situational report.
The general responsible for iraq leans forward and says, “Mr President, I have bad news. 7 Brazilian troops were killed by road side bombs last night in Iraq.”
The President is stunned. After a few minutes he manages to compose himself sufficiently to ask, “General, that’s terrible! Tell me, exactly how many zeroes are there in a brazilian?”
Like the President the average corporate should not be focusing on the zeroes.
The value of big data analytics, for most companies, will be in the ability to gain insight in existing data that is currently not being exploited – so called dark data. This data is of a reasonable size – probably running into the tens or hundreds of terabytes – rather than being super large. Very few banks, retailers or insurance giants will even be thinking in terms of the large volume of storage punted in the presentations. [Tweet this]
Yes, Hadoop can bring economies of scale and allow companies to store and analyse more data than is currently the case.
For example, one telecommunications company can now run analytics across 11 years of Call Data Records (CDRs) – the basic unit of measure for telecommunications billing – rather than their previous EDW which could only store 3 months worth of data for the same cost. This allows them to build more accurate fraud and churn models, simply because they have access to more content. But the volumes are defined, not meaningless.
Let’s top throwing meaningless numbers at the world and start looking at real case studies. The numbers are far less alarming than the hype suggests!
Check out some real world big data use cases here!
Image sourced from http://upload.wikimedia.org/wikipedia/commons/3/38/Rio_de_Janeiro_Helicoptero_49_Feb_2006_zoom.jpg