What is Big Data

“From the dawn of civilization until 2003, humankind generated five exabytes of data. Now we produce five exabytes every two days…and the pace is accelerating.”Eric Schmidt

Big data refers to our ability to collect and analyze the vast amounts of data we are now generating in the world. The ability to harness the ever-expanding amounts of data is completely transforming our ability to understand the world and everything within it. The advances in analyzing big data allow us to e.g. decode human DNA in minutes, find cures for cancer, accurately predict human behavior, foil terrorist attacks, pinpoint marketing efforts and prevent diseases.

In ‘Big Data’ world we have 4 Vs that characterize big data:

  • Volume – the vast amounts of data generated every second
  • Velocity – the speed at which new data is generated and moves around
  • Variety – the increasingly different types of data (ex: financial data, social media feeds, photos, sensor data..)
  • Veracity – the messiness of the data (ex: Twitter posts with hash tags, abbreviations, typos and colloquial speech)

Some More Real World Examples of big data:

  • The FBI US is combining data from social media, CCTV cameras, phone calls and texts to track down criminals.
  • Facebook is using face recognition tools to compare your photos with those of others to find potential friends.
  • Politicians are using social media analytics to determine where they need to campaign more to win.
  • Video analytics & sensor data of Baseball or Football games is used to improve performance of players & teams.
  • Google’s self-driving car is analyzing large amount of data from sensor & cameras in real time to stay on road safely.
  • The GPS information on where our phone is and how fast it is moving is now used to provide live traffic up-dates.
  • Companies are using sentiment analysis of social media posts to determine and predict sales & brand equity.
  • Supermarkets are using loyalty card data with social media information to detect & leverage buying patterns.
  • A hospital that looks after sick babies is generating a live steam of every heartbeat, to identify patterns.