Big Data is Ubiquitous

We are living in the era of Big Data, where a wealth of data is being generated every minute, every second, even every nano second. Thanks to the astounding technological revolution, everything around us is being captured in some way or the other, stored in some form, and this has the potential to make business sense. Big Data refers to any data set that is too big to be efficiently worked in real-time with traditional database tools. Modern businesses generate huge volumes of structured and unstructured data in both traditional and digital forms. Many firms have realised that these data could hold promise to give deeper insights into their customers, partners, and other stakeholders.

Source: mashable.com

Businesses generate huge volume of data in multiple forms – both structured and unstructured:

• Structured data include traditional data.
• Unstructured data include mostly non-traditional sources such as blogs, social media, email, sensors, photographs, video footage, GPS information via mobile devices, etc.

Big data presentations in leading conferences such as the Big Data Analytics Conference 2012 in London highlight that every day we send approximately 300 billion e-mails and share one billion items on Facebook; every minute we post 170,000 tweets to Twitter, 3,000 photos to Flickr and 48 hours of video to YouTube (presentation by e-Cognosys). Approximately 70% of all data is created by individuals – their photos, blog posts, music playlists, purchases, Facebook posts, tweets, Skype calls, online bank transactions, and so on.

Big data is ubiquitous. Its remit spans:

• health care (e.g., monitoring patients remotely, gene sequencing)
• manufacturing (e.g., data on machine health using sensors)
• logistics and supply chains (both historical - demand data, route data, TMS, WMS, ERP - and real time - through sensors including “internet of things”, RFID, GPS, QR codes, social networks)
• utilities (e.g., smart meters)
• social data (Twitter, Facebook, etc.)
• policing and inland security (e.g., Fraud detection, tax audits)
• e-commerce
• financial services (proactive customer engagement, Risk & Portfolio Analysis, Investment Recommendations), and
• education (student engagement monitoring, attendance, etc.).

Any talk on Big Data invariably includes 4Vs – volume, variety, velocity and value. Big data is huge in volume, has many varieties as mentioned above, it is being generated at a rapid rate, and has the potential to generate value to businesses. In addition, there are two more Vs that are not often discussed – visibility and variation. Not all data is visible to every one; some are public while many are private data. There are huge variations in the quality of data available in different forms. This variation in quality is one of the most serious limitations to the use of Big Data. For example, scientificdata similar to those available in CERN may have consistently good quality compared to social data.

Data, analytics and intelligence play important roles in exploiting the power of Big Data (see the UoB visual above). The real challenge is to understand the barriers to adopting big data and identify ways to overcome these barriers. Timely and sufficient investment in human skills is critical – much more critical than technological investments. Avoiding “analtytics paralysis” is important.

About the Author: Prof Ramakrishnan Ramanathan

Ram is Professor of Operations Management and Director, Business and Management Research Institute (BMRI). Last year, Ram was awarded a prestigious award from Emerald Literati Network for an article he co-wrote with Dr Usha Ramanathan.