Big Data only for Big Companies?

I have started discussing this topic with some people at the Swiss Association for Analytics event in March.

If you haven’t been living in a cave for the last few years, you are aware of the new buzz word in our field: Big Data. I’m not a big fan of this term. Indeed, the real challenge with high volume of diversified and (un)structured data is to make sense of them. This is why I prefer terms such as data mining and analytics. Anyway, for more information about the three “D” of Big Data, have a look at the end of this post.

My main concern about Big Data is not the term used, but its current application within industry. Although a lot is written about Big Data, I think that very few companies really use Big Data. Of course, analytical competitors are fully using and benefiting from Big Data: companies such as LinkedIn, Facebook, Twitter, Google, Amazon, etc. Most of them are both part of it and use it. Although huge, these companies only represent a small percentage of all industries using analytics.

bdtrends
Google Trends for “Big Data” from 2011 until now.

At least in Europe, most companies are not yet at the level of Big Data. Most of these companies (whether medium or large) are yet working on data quality and business intelligence (reporting). Some of them are just starting to use data mining. It is already challenging enough for these companies to leverage structured data. Don’t get me wrong: I’m not against Big Data. However, I think companies should not start Big Data projects by taking data (noise?) from any possible sources, just because some key players have Big Data projects (with data they generated, for example). In conclusion, I think that the term is badly chosen and the discussion level is far above reasonable, compared to the real use by the industry. But wait, I just added to the discussion with this post…damned!

What do you think? Out of the term itself, do you think the buzz around Big Data is valid? Fore more information, see SAS and IBM definition of big data.

Share

Recommended Reading

Comments Icon4 comments found on “Big Data only for Big Companies?

  1. I definitely agree. Few companies have Big Data or few really need to analyze such big amount of data. Big Data does not mean Big Information nor Big Knowledge.
    A Hadoop cluster or an in-memory database do not fix any unclear business requirement or substitute a clear goal of a data analysis.

  2. Also of concern is how much data is collected by private sources that cannot be accessed by researchers. There are efforts such as The Collaborative Social Media Observatory that are trying to remedy this, but we (researchers) are really on the backfoot already in such endeavours given the commercial value (overvalue?) placed on such data, and that public institutions cannot afford to purchase this data.

  3. Sandro, with the explosion of big data, companies are faced with data challenges in three different areas. First, you know the type of results you want from your data but it’s computationally difficult to obtain. Second, you know the questions to ask but struggle with the answers and need to do data mining to help find those answers. And third is in the area of data exploration where you need to reveal the unknowns and look through the data for patterns and hidden relationships. The open source HPCC Systems big data processing platform can help companies with these challenges by deriving insights from massive data sets quick and simple. Designed by data scientists, it is a complete integrated solution from data ingestion and data processing to data delivery. More info at http://hpccsystems.com

Comments are closed.