From Flawed Assumptions to Accurate Insights: The Power of Store-Level Data and AI in Retail

August 7, 2023
By Heather Martin, VP Marketing & Sales, HIVERY

In the volatile world of retail, Consumer Packaged Goods (CPG) companies and retailers are constantly balancing shopper needs, affordability, profit, and category expansion. Traditional approaches, such as consumer research and shopper panel data, have long been the go-to methods for understanding shopper behavior. However, there’s an unspoken truth that the industry needs to confront: Shoppers lie because they are aspirational, and therefore the data we gather from them is at best incomplete and at worst misleading.

Take toothpaste as an example. Market research and resulting consumer decision trees might suggest that consumers desire toothpaste that freshens breath, whitens teeth, fights cavities, and tastes good over price, but do these stated preferences translate to decisions at the shelf? A shopper in a dollar store might choose a less expensive product despite lacking some of the features they initially wanted. Moreover, shopping patterns within the same channel often vary for the same shopper and category, influenced by unique store offerings, from price to range to pack size. Gone are the days when brands could apply a uniform strategy across a single channel. This disparity drives a need for more granular research, which becomes too much to analyze and is discarded in favor of a high-level strategy that can be executed.

This is not new – leading retailers and CPGs acknowledge the limitations of current research methods. In Walmart’s most recent annual report, it was highlighted that the success of their business depends in part on how accurately they predict consumer demand and availability of merchandise:

“It is difficult to predict consistently and successfully the products and services our customers will demand and changes in their shopping patterns ” – 2023 Walmart Annual Report, page 15.

Their report highlights that pricing and merchandising strategies partly influence their impact on net sales and gross profit margin in response to cost increases. The importance of responding to changing consumer tastes, preferences, and shopping patterns, stating that failure to do so could negatively affect their reputation, customer relationship, product demand, market share, and business growth.

I’m not picking on Walmart – most retailers and CPGs highlight similar risks to Wall Street. It’s clear that finding a better, more consistent, data-driven approach to support effective decision-making, managing inventory levels, dealing with supply chain disruptions, and driving the right assortment to the right store at the right time is vital to the continued success of retailers and brands.

The critical point is this: today’s ever-changing market requires a retailer-specific approach but illustrates that demographics and traditional research methods are insufficient. Even when they are, they fail to provide the ‘how’ to execute and capture the growth opportunity. We have already seen that demographics alone are not always the best indicator of consumer preferences, and companies like Amazon and Netflix transformed how we consume by focusing on what the data about each customer’s past consumption suggests about their future needs, regardless of location or demographics.

Is there a better way in retail? What if you could understand shopping patterns at the store level better and easier? With powerful AI algorithms, leveraging store-level data, retailers and CPG manufacturers can model what we call “The Chameleon Effect” – the shifting shopper behavior across retailers.  This retailer-specific approach ensures retailers and CPG brands can make decisions that deliver locally relevant, effectively merchandised, and operationally efficient assortment plans. With store-level insights, you can achieve this strategy far more effectively, uncovering new growth opportunities.

We are seeing CPGs and retailers realize that traditional research methods have limitations in informing decisions about portfolio and category assortment. Instead, they use AI and store-level data to augment their decision-making processes. Organizations can gain more precise and actionable insights and test hypotheses before executing in-store by adopting a store-level data analysis approach. This leads to better decisions, improved performance, and a better shopper experience. With the democratization of data and AI, it’s now possible to derive insights into consumer behavior and shopper preferences by analyzing store-level point-of-sale data.

So where does this lead us? The retail industry must embrace that traditional research methods are limited, and relying solely on them can lead to flawed business decisions or analysis paralysis. By leveraging technology and data, we can revolutionize how we approach decision-making across the assortment lifecycle, leading to more accurate, effective, and executable strategies.

It’s one of the reasons I joined HIVERY – our vision to harness the transformative potential of data to unlock new possibilities is behind our belief that Data Has A Better Idea™. Rather than relying on top-down processes and averages of averages, AI allows CPG brands to tailor assortment portfolios to each retailer, considering unique shopper purchase behavior and that retailer’s unique execution strategy without the humanly impossible task of analyzing millions of variables, combinations, and patterns.