Future Forces: Big data dilemma

To survive in tomorrow’s world, your internal rate of change must be greater than the external rate of change around you. No industry has more shopper behavior data than grocery and CPG companies. Since the invention of UPC symbology in the 1970s, retailers and manufacturers have tried to understand consumer/shopper behavior through in-store sales analysis, loyalty analysis and external market research. Industry success is now measured through more than 45,000 items in a typical supermarket trying to find space for some of the 15,000 new items added each year to serve American households that purchase the same 150 items each month that represent 85% of their needs. Now entering our world of data comes 1,500 blogs, 98,000 tweets and 168 million emails every minute of every day. This flood of data is created by the interactions of millions of people using smart phones, GPS devices and Internet conversations. Capturing these massive amounts of data and turning it into actionable information that identify consumer wants and needs is the promise of “big data.” Cloud-based computing platforms will capture online consumer interactions that enhance our ability to understand shopper purchase behavior. Pieces of this future disruptive model already exist under names such as Shelf Snap (intelligent photos), Gigwalk (crowd sourcing), ShopKick and MobileCoupons.com, as well as web scraping and viral buzz technologies, with many more apps appearing each month. Yet, despite white papers from the 2012 World Economic Forum in Davos, Switzerland and industry research reported at the recent GMA Executive Conference, how to harness big data remains an ill-defined promise of artificial intelligence with few best practices and unproven business value. Do not get me wrong. We can all cite examples of successful data mining and can see the growing sophistication of shopper apps. No, my concern is that we first need to ask what problems are we trying to solve that have big business value. When will the industry tackle big business problems such as excess inventories, out-of-stocks, channel migration and center store sales declines?  The question we should be asking ourselves is can we do better as part of an industry that has had limited success in shared shopper knowledge from common databases?  Mobile devices may be allowing the amount of data in the world to explode, but our dilemma revolves around our ability to change the culture of the industry. The pathology of our industry is risk avoidance. We have had multiple opportunities as an industry to embrace change. From Walmart’s RetailLink in the ‘80s, to ECR in the ‘90s to global data exchanges such as UCCnet and 1Sync. CPG companies built their production systems independent of demand-side retailer data. Our legacy systems of long production runs on the country’s oldest production equipment require a push deal model to create sales. Likewise, retailer profits require taking advantage of this push model with more than 50 extra profit centers. Big data collaboration will require a level of trust, in the data, systems, commercial value and core competency not yet demonstrated between most trading partners. We can all see this disruptive change coming in other industries with company names like Circuit City, Best Buy, Borders and Radio Shack. Tomorrow’s environment requires business intelligence and analysis that provides an end-to-end view of the demand cycle. It does not matter if you call it big data, distribution without waste or field-to-shelf visibility. Your competitors of the future are building this model. Yet, most companies do not have the time, talent or treasure to give up their old profit models while simultaneously establishing a new collaborative consumer demand enterprise even if it means a better ROI on trade spending, reduced supply chain inventories and improved on-time deliveries with fewer out-of-stocks. Think about this on your drive home tonight. To survive in tomorrow’s world, your internal rate of change must be greater than the external rate of change around you. Next generation collaboration is already breaking down the silos around organization functions. These new models are not enslaved to traditional data aggregators or profit centers. New companies are reinventing the merchant culture with partners beyond our traditional industry. How will we keep pace? Patrick Kiernan, managing partner of Day/Kiernan & Associates, is affiliated with The Center for Food Marketing at St. Joseph’s Uni­versity, Philadelphia; the Institute for the Future, Palo Alto, Calif.; and Encore Associates, San Ramon, Calif. He can be reached at [email protected].