Starting with the Basics: Big Data 101

Before diving into specific industry implications, it is important to level-set with a commonly accepted and industry-generic understanding of “What is Big Data?”

Big Data is a term that can be described in several ways. The most commonly used description of Big Data references “The Three V’s” – Volume, Velocity and Variety. Typically, only one of these attributes needs to be present for a data asset to be considered Big Data; however, often two or all three attributes co-exist.  The following is a brief description of the three V’s:


Volume refers to the sheer size of the data asset being analyzed. While many traditional data assets were once measured in gigabytes, today, exploring terabyte and even petabyte data assets is becoming more common.  Many CPG companies have reached this volume threshold simply as a result of the accumulation of historically “small” (or existing and traditional) data assets such as point-of-sale (POS) transactions. However, many new emerging data assets, by their very nature, bring unprecedented data volumes to the industry. 


Velocity refers to the speed at which data is being created and the need for the business to consume it.  Velocity of data has seen an exponential rise in recent years with emerging technological and digital innovations such as the proliferation of consumer apps, enhanced sensors, GPS-enabled devices, and objects connected to the internet (“The Internet of Things”).


Variety refers to diversity in both the source and format of the data. Technological and digital innovations are creating new sources of data outside, or external to, the organization versus the traditional internal sources.  The format of this data is trending away from traditional structured data that fits nicely into rows and columns, to data that is becoming progressively unstructured in the form of text, video, audio, graphics, and photos.

Categorizing Big Data: Internal vs. External, Structured vs. Unstructured

From a CPG perspective, categorizing Big Data across two key dimensions, Internal / External and Structured / Unstructured, is one way to better understand the emerging Big Data landscape. In the chart below, the notion of traditional “small” data can be represented by the bottom-left quadrant of Internal / Structured. However, even this data can eventually become “Big” as historical data accumulates resulting in larger volumes. 


The other three quadrants are more representative of the Big Data that is impacting the industry. This data comes from the growing volume of existing data (e.g., POS, shopper profile, etc.) as well as emerging forms of data fueled by technological and digital innovations (e.g., mobile eCoupons, sensor tags, video, text, photos, audio). The explosion of unstructured and external data is what will lead organizations to rethink how they approach their next evolution of analytical capabilities.

Hype vs. Reality

The hype around Big Data began to snowball in August 2010 when Google CEO Eric Schmidt, stated, "There were five exabytes of information created between the dawn of civilization through 2003, but that much information is now created every two days, and the pace is increasing...People aren't ready for the technology revolution that's going to happen to them."[1]

Schmidt’s comments aroused interest in Big Data as a new concept that had the potential to disrupt business as usual. However, considering references to the “dawn of civilization” and the quantifier “exabytes” – which raises eyebrows but doesn’t have tangible industry relevance – this statement tends toward the hype category.

Even so, the hype appears not to be too far removed from reality. Fast-forward to January 2012 at the World Economic Forum held in Davos, Switzerland. At this conference, global leaders in economics and business convened to discuss the potential impact of Big Data. The discussions focused on how the emerging era of technological explosion could spawn Big Data and result in unprecedented opportunities for those prepared to harness it.  Participants compared Big Data to other factors of production such as human resources, physical assets, and intangible assets such as trademarks and copyrights. The forum ultimately acknowledged “data as a new class of economic asset,” with properties similar to “currency or gold.”[2]

“Currency or Gold”, “The New Oil”?  These are strong comparisons.  Deloitte assembled a panel of leading Consumer Products and Retail Financial Analysts to share their thoughts, and Wall-Street’s perspective, of investor perception around this topic of Big Data Hype vs. Reality]

In the video, Deloitte discusses with a panel of Wall-Street industry analysts the following topics:

  • Perspective on why the investment community should pay attention to a company’s ability to leverage Big Data

  • Perspective on analytical maturity of CPG and Retail, and how it has evolved with Big Data

  • Examples in which companies are gaining competitive advantage by using Big Data and Analytics to drive category growth

  • Readiness of the industry to take advantage of Big Data and Analytics to drive growth

  • Factors holding companies back from capitalizing on Big Data and Analytics

  • Perspective on who owns the development of the analytical maturity of an organization

  • View on analysts’ expectations of the value of Big Data and Analytics for CPG and Retail

The growing opinion that Big Data presents tremendous opportunities should not be ignored – especially by the CPG industry.  Consider the following estimates:

  • By 2016, half the world’s population, more than 3 billion people, will be online, and the majority will be using mobile devices.  That is up from 746 million in 2005.[3]

  • Mobile retail revenue – purchases made using an “app” – is expected to increase from $6 billion in 2012 to $31 billion by 2016.[4]

  • It is estimated that a leading global retailer collects more than 2.5 petabytes of customer transaction data every hour.  That is equivalent to about 50 million filing cabinets worth of text.[5]

But what is the value proposition to the CPG industry? What makes data so “Big” for the CPG industry? Without clarity on the value of Big Data for consumer products companies, Big Data’s buzz may be prematurely written off as an over-promised, under-delivered technology trend. The companies that mistakenly fall into this line of thinking could be at a disadvantage early on. As technological and digital innovation continues to progress at exponential rates in contrast to the linear pace of change within the industry to date, a competitive divide may be emerging in the industry. 

The Reality – 5 Summary Conclusions and Recommendations to the Industry

The reality is that to take advantage of Big Data, CPG companies need the right people with the right skills and talents to improve the quality of decision making. Big Data in itself is not a solution, but simply an input and enabler to becoming a better informed, more productive organization. All of the investment in Big Data and analytics is wasted if decision makers feel the insights they receive are erroneous. There is also little value in having Big Data insights in the hands of decision makers who lack the skills and competencies to derive the proper business decisions.

To extract value from Big Data, CPG companies will have to stay focused on the end-game — the ability to make more accurate information-based decisions that drive improved organizational effectiveness and business performance. In addition, and perhaps not as obvious, CPG companies should build a broad set of capabilities and competencies around Big Data and analytics before they can partake in – and capitalize on – the rapid pace of technological and digital innovation.

The following five key conclusions and recommendations are a further explanation of the “reality” of Big Data and summarize the findings from the broader research study:

1.   The majority of the CPG industry is lagging in data and analytical capabilities

Over the last decade, the majority of firms in the CPG industry have not progressed beyond localized analytics and are still challenged to capitalize on existing data.

2.   Rapid-fire pace of innovation requires data & analytics competency

The rapid-fire pace of technological and digital innovation has the potential to define winners and losers in the CPG industry, and an integrated core competency around small data, Big Data and analytics is a requirement for the winners.

3.   The CPG industry is moving from linear change to exponential disruption

Until now, a CPG company’s competitive advantage did not depend on having strong analytical maturity; however, the industry’s preconceived notion of change is about to be disrupted by the exponential pace of innovation. Navigating this landscape will require companies to harness data and efficiently convert it to insights.

4.   Business context is required to operationalize big data, analytics, and innovation

The Formula for Growth is most impactful when applied in the context of cross-functional and integrated business planning and execution.  Whether small or Big Data, ultimately the goal is to consistently make better decisions to improve the organization’s planning and execution toward achieving top and bottom-line growth objectives. 

5.   CPG companies need data and analytics to support an innovation-driven culture

This new “exponential” era the industry is entering will require a culture of experimentation, trial and error, and other methods that support a company’s ability to remain agile, fluid and adaptable to the pace of change.  A strong versus weak data and analytical maturity is one of the key differentiators between a CPG company that will successfully navigate through this new era of innovation, versus falling victim to the exponential pace of change.

Deloitte’s Marcus Shingles and, Tom Davenport (Harvard Business School and Best-Selling Author on Analytics), topline and discuss the five key conclusions and recommendations from the GMA industry research Formula for Growth – (Innovation)Big Data & Analytics


[1]Statement made August 4, 2010 by Eric Schmidt at the Techonomy conference in Lake Tahoe, CA. http://techcrunch.com/2010/08/04/schmidt-data/

[2]The New York Times: “The Age of Big Data,” Steve Lohr, Published: February 11, 2012.

[3]Harvard Business Review: “It Keeps Growing…and Growing”, Published: October 2012

[4]Forrester: “Mobile Is The New Face of Engagement”, by Ted Schadler and John C. McCarthy, Published: March 26, 2012

[5]Harvard Business Review: “Big Data: The Management Revolution”, Andrew McAfee and Erik Brynjolfsson, Published: October 2012