The Ultimate Guide to Demand Forecasting in Grocery Retail (2024)
This guide is a tool to help retailers build a foundation of modern demand forecasting. Feel free to navigate through it at your own pace using the table of contents.
Table of contents
- What Is Demand Forecasting?
- Need for Demand Forecasting in Grocery Retail
- Why Care About Demand Forecasting Now?
- Demand Sensing vs. Demand Forecasting: What’s the Difference?
- Why Most Forecasting Frameworks Might Be Obsolete for Grocery Retail
- A Framework for Effective Demand Forecasting
- What Is the Role of AI in Demand Forecasting?
- Why AI/ML Is a Game Changer in Demand Forecasting
- 5 Key Benefits of AI-powered Demand Forecasting
- 5 Important Features to Consider in a Demand Forecasting Solution
- Future-Proof Your Retail Forecasting
Retailers today face increasing complexity in fresher products, elevated levels of omnichannel expectation, and orchestrating supply chain to manage unanticipated demand and supply. Most grocery retailers have identified demand forecasting as a key focus area to deal with the challenges as it is the single most important activity that determines the success of category, merchandise, and supply chain management.
Gartner suggests that retailers focusing on demand forecasting experienced a 32% reduction in out-of-stock items and a 20% improvement in inventory levels optimization.
In this guide, we delve deeper into the concept of demand forecasting, its benefits, and key features that your forecasting framework should have to succeed in grocery retail.
What Is Demand Forecasting?
Demand forecasting is the process of making future estimates of customer demand over a specific period at a given price point, in a specific time frame and location.
Generally, demand forecasting considers historical data and other analytical information to produce the most accurate predictions.
However, in grocery retail, demand forecasting requires a granular approach to prediction — monthly, weekly, or daily forecasts to support various category management processes and supply chain decisions. Granular and adaptive forecasts are critical, given the wide spectrum of products varying from fresh to ambient products.
Need for Demand Forecasting in Grocery Retail
Grocery retailers today face unprecedented top and bottom line pressure due to an irrevocably changed grocery environment. Globally, grocers lose about $1 trillion in lost sales due to out of stock (OOS) in a year. Additionally, $750 billion worth of food products are wasted every year.
Moreover, Mckinsey’s research suggests that consumers are challenging brand loyalty by changing brands frequently. 36% of consumers are trying a new brand while 25% have moved to private labels.
Furthermore, omnichannel grocery spend is slated to grow by 18% annually.
The enduring nature of these challenges are an indication that grocery retailers need to catch up with evolving customer preferences and strengthen their supply chain to ensure that shelves are properly stocked – in stores and online. It has become increasingly important to get demand forecasting right down to a T.
Gaps in On-Shelf Availability (OSA), both online and in-store, can be catastrophic and result in lost volume, lost baskets, and lost faith among shoppers.
Demand forecasting is also a very strategic piece of the puzzle in a retail organization. It has a direct effect on the financial health of a retail organization – profit margin, cash flow, working capital, and operational expenses are some of the key business metrics that it affects. Moreover, it is one of the key enablers of growth and market expansion.
Here are some of the key processes that rely on accurate demand forecasting:
Supply Chain Optimization
Demand forecasting forms the backbone of supply chain decisions such as location, production, inventory, and transportation. By accurate forecasting, supply chain planners can optimize their supply chain to reduce cost, bring in efficiency and responsiveness.
Cost Efficiency and Financial Planning
A strategic emphasis on demand forecasting goes a long way in increasing your cost efficiency and making your business profitable. By planning for trends, peak intervals, seasonal demands, and accounting for supply side risks, grocers can optimize working capital and maximize cash flow.
Implement Relevant Marketing Campaigns
With a clear understanding of customer demand, you can make your marketing efforts more targeted, cost-effective, and tailored to your demand curve. Proactive marketing interventions can help you boost sales in lean periods and achieve greater marketing ROI throughout the year.
For example, in the case of reduced sales prediction, running appealing promotional offers and discounts for a limited period will increase the purchase, to balance out the potential loss.
Improved Customer Experience
Customer experience is the key to every business, especially post-pandemic. Consumers value price and convenience over anything else.
Research suggests that 75% of consumers are modifying their purchase behavior to accommodate economic changes, store closings, and evolving priorities.
By understanding customer trends and anticipating customer needs, category managers can stock the products that customers are most likely to want, when and where they want them. This prevents out-of-stocks and the customer dissatisfaction that comes along.
Addressing Staffing Issues
Staffing issues are visible during both peak season and a dip in demand. While lack of staff can result in customers waiting to get a service, the availability of more staff can be overwhelming. Therefore, predicting retail demand will allow managers to schedule staff shifts accordingly for an elevated customer experience.
Why Care About Demand Forecasting Now?
- Research suggests that loss of sales has increased from 5-10% historically to 15% post-pandemic
- Loss due to food wastage is recorded at 30% due to retail store inefficiencies and agricultural waste. The World Bank estimates per person per day food wastage to reach 2 kg, by 2025.
- The cost of holding excess inventory in grocery retail has ballooned up to 30%, reducing the profit margins significantly
- A retailer risks losing up to 10-30% of its revenue if it fails to cater to the omnichannel demand
Demand Sensing vs. Demand Forecasting: What’s the Difference?
Category managers and supply chain managers would ideally like a 100% accuracy in demand forecasting. But the problem is that it does not exist yet due to the vagaries in the business. Digital technologies such as artificial intelligence and machine learning can elevate demand forecasting accuracy up to a certain limit.
Demand forecasting works really well for mid to long-term planning. However, it needs to be complemented with effective short-term planning. This is where demand sensing comes into play.
Demand sensing is a more short-term focused forecasting approach that predicts customer demand on a daily and weekly basis. It leverages ML/AI to factor in a wide range of demand predictors that factor in real-world events that affect customer behavior.
With the help of demand sensing capabilities, grocers can now more effectively determine where, when, and what products consumers will buy on an ongoing basis. It helps build supply chain resilience, improve inventory management, and better estimate demand.
Source: LinkedIn
Why Most Forecasting Frameworks Might Be Obsolete for Grocery Retail
The conventional forecasting frameworks are proving to be inefficient because of the following key reasons:
One-size-fits-all approach
Traditional forecasting methods use predefined rules and are unable to capture the diversity of the complex product matrix. Every product category and distribution channel has unique requirements and needs a tailored approach.
Static one-time forecasts
Traditional forecasts are unable to account for internal factors dictating the aisle. For example, the forecasts are not considering the markdowns, promotional offers, and short-term cannibalization occurring due to the unavailability of competing products.
Dependence on manual interventions
In the case of sparse data, noisy data, and shorter product life cycles, the traditional forecasting accuracy relies on manual intervention, making it prone to error.
Lack of a recommendation engine
Demand forecasting is a highly intensive process with a series of decisions that determine forecasting accuracy. Most of the current tools do a great job of analyzing and visualizing data. However, they miss out on enabling the category managers and supply chain planners to make the right decisions, such as what predictors to use for what category, what model to use given the data, what data to exclude for analysis, etc.
eBook
A Framework for Effective Demand Forecasting
100% forecast accuracy is perfect, obviously, but in most cases it is far away from the perfect score, and that is alright. Anything above 70% is considered acceptable.
However, measuring forecast effectiveness through accuracy is a very narrow view due to two reasons: it is only an absolute value of error, and it does not tie into category objectives. For example, it is okay to overestimate sales of ambient products as opposed to fresh products where there is considerable spoilage.
To achieve greater accuracy while meeting category objectives, three critical capabilities need to be a part of your forecasting framework.
Data Capturing and Enrichment
The demand forecasting framework must capture the myriad forms of data generated across the retail organization apart from historical sales data such as discounts, inventory, and promotions as well as important external factors such as weather and macroeconomic indicators. The framework also needs to be robust enough to tackle modern data challenges such as sparse and noisy data.
Tailored Forecasting
The forecasting framework needs to seamlessly tailor forecasts to channel, store, and category nuances.
For example, the demand for fresh and seasonal products in a particular store would be different from the demand for an ambient product in the online channel. Identifying patterns in each of these combinations and tailoring forecasts to it can go a long way in optimizing stock and winning customers.
Agile Forecasting or Demand Sensing
Forecasting should instill agility in category management. Predominantly, customers react to what is happening on the shelf. If your category management team can react to them preemptively, then that will be a huge win for your retail organization.
For example, effects like cannibalization, out-of-stock, and halo effect can be simulated way in advance, avoiding last-minute panic to stock shelves. The forecasting framework needs to be the go-to tool for category managers to be able to do all of the above-mentioned tasks in a few clicks.
What Is the Role of AI in Demand Forecasting?
Demand forecasting typically is a statistically driven process that requires the planner to make a series of decisions to arrive at the final forecast which forms the basis of key decisions across category management and supply chain management.
Traditionally, planners have been effective in using existing solutions in ERP and analytical tools to achieve a reasonably accurate forecast with a tolerable forecasting error. However, in recent years, most retailers have experienced a large forecasting error as grocery retail witnessed a seismic shift globally, partly accelerated by the pandemic.
The #1 reason for the large forecasting error has been the inability of traditional static forecasting frameworks to help the planner make the right decisions while forecasting. Category managers need to factor in the right indicators of demand and right models for each channel, store, and category combination to achieve accurate forecasting.
This is where AI comes in. While AI is a grossly misused term across industries, in grocery retail, specialized AI has proven to be highly effective in producing accurate forecasts every time. Customers who have adopted it have reported improvement in forecasting for 90% of the SKUs.
Why AI/ML Is a Game Changer in Demand Forecasting
Treating Data Challenges
Your forecast is only as good as your data. Still, most retailers attest to it that even in a sea of data that they have at their disposal, they often find themselves struggling with noisy and sparse data. They either fail to spot these challenges, in which case their forecasting is jeopardized, or depend on external teams to get the data cleaned which is a time-consuming process.
AI can help solve these challenges with ease. It can treat noisy data with machine learning techniques that can identify outliers and invalid data. Whereas, in case of sparse data challenges, attribute based forecasting can bridge the gap and provide accurate demand curves for new products. All of this can be achieved without the help of any external team.
Enriching Data with Real Predictors of Demand
Traditional frameworks use just historical sales data to create forecasts. This approach, however, has a major flaw – it assumes that future demand patterns will be more or less similar to past demand. It completely ignores the causality aspect.
For example, let us consider weather as a predictor of sales. For categories such as cold beverages and desserts, there is a strong causal relationship between temperature and sales. So if during the forecasting period, temperature is higher on an average compared to last year, then the classical forecasting framework will fail drastically.
The factors that impact demand range from external factors such as weather, holidays, macroeconomic factors, etc. to internal factors such as pricing, inventory etc.
Incorporating this wide range of information in your forecast is a huge challenge simply because of the complexity of data. Thankfully, with machine learning and easy access to computation capabilities, you can factor in a wide range of predictors and relationships that impact demand on a daily basis into your retail forecasts with just a few clicks.
Selecting Category, Channel, Store-specific Predictors
Demand planners often face the dilemma of choosing the right predictors for each category/product. Oftentimes this decision is made on the basis of past understanding of how the category behaves or even worse with a one-size-fits-all approach.
For example, while demand for meat products goes up sharply during weekends and holidays, demand for chilled beverages is highly correlated with weather. Using the same approach for both the categories may prove disastrous, given the stark difference in the predictors for the categories.
This is where machine learning can do the heavy lifting. It can recommend what predictors are significant indicators of demand for a channel, store and category combination. This goes a long way in improving the overall forecasting accuracy.
Finding The Best-Fit Model To Meet Your Category Objectives
Each channel, store, and category behaves differently. There is no “silver bullet” model that can give you the best forecast.
For example, for a fresh product in a small remote store, the selection of the forecasting model should be biased towards reducing wastage. On the other hand, for an ambient product such as canned food in a high footfall store, the selection of the forecasting model should be biased towards reducing out-of-stock instances.
Typically such optimizations are achieved with frequent manual interventions over time, but they are prone to errors and often complete neglect. With AI, however, model selection can be done automatically based on best-fit in line with your category objectives.
Forecasting with advanced techniques such as ML and deep learning, and ensemble based approach of combining multiple models, has proven to be much more accurate than traditional forecasting frameworks.
Integrating with Customer Data Systems and Curating Attractive Assortment
There has always been a greater impetus in grocery retail on understanding what a customer truly desires.
With the ever-evolving nature of customer data systems that capture a variety of data—from social media, eCommerce site, app, POS, etc.—it has become imperative to integrate this information in your demand planning, so your shelves appear attractive and fresh to customers every time they visit.
With traditional forecasting frameworks, it is near impossible to integrate this information into your demand planning due to the sheer complexity of depth and width of categories.
But with digital technologies such as AI/ML, all this information can be fed seamlessly into your demand planning system. The outcome – your customers get a delightful shopping experience customized as per their preferences.
5 Key Benefits of AI-powered Demand Forecasting
AI has already made a mark in demand forecasting, and the results are promising. Here are the key benefits offered by AI-powered demand forecasting solutions such as Algonomy’s Forecast Right:
Improvements in Data Forecasting Accuracy
Over time, machine learning algorithms learn from the historical data trends and with more data being fed to the system, the data forecasts become better. Any new emerging pattern in demand is automatically fed into the ML-engine, thereby making it more robust and adaptive.
Moreover, the AI-assisted features help planners with data treatment, feature engineering and selection. As a result, 90% of categories show instant improvement in accuracy as compared to traditional time series-based forecasting.
Effective Replenishment
With accurate SKU level forecasts and retail-tuned replenishment optimization solutions such as Algonomy’s Order Right, retailers have been able to reduce out of stock instances by 75%, wastage by 30%, and inventory cost by 10%.
Order Right optimizes order plans for your supply chain constraints such as lead time, minimum order quantity, and category factors such as expiration date, etc.
Increased Customer Satisfaction
Nothing is more frustrating for a customer than out-of-stock and expired products on the shelf. Demand-driven forecasting and supply chain-optimized replenishment leads to improved shelf availability.
In addition, demand sensing capabilities help category managers catch on to emerging trends before competitors and eliminate non-performing SKUs, making sure that shelves appeal to customers.
Improved Workforce Productivity
Your category managers show improved productivity when they spend more time on strategic aspects of improving margins and achieving growth rather than working on difficult-to-use dashboards and tools.
Using an AI-based demand forecasting solution that is business-user friendly has proven to make the entire forecasting process up to 100x faster.
Effective Markdown/Discount Optimization
Retailers often incur higher inventory holding costs when products remain unsold for longer periods. There’s a risk of products hitting their expiry date and losing value. Ultimately, the retailers have to sell these products at a discounted price leading to margin erosion.
This scenario is greatly minimized with an AI-based forecasting framework which not only reduces forecasting errors but also builds demand sensing capabilities which allows category managers to react to scenarios playing out on shelf in a swift fashion.
5 Important Features to Consider in a Demand Forecasting Solution
With a wide variety of demand forecasting solutions in the market, it becomes overwhelming to narrow your search to a single solution that can help you with your business goals. Here are the top features to look for in a demand forecasting solution:
Demand Indicators
Your solution must account for the full spectrum of demand forces such as historical trends, external factors such as weather conditions, macroeconomic factors, etc., and internal factors such as promotions, pricing etc., without tedious configurations that require frequent tech support.
Retail-tuned ML Algorithms
Since each store, category, and product is different, the solution must use an ensemble of algorithms with a built-in feature engineering and selection engine. The focus should be on achieving category objectives rather than simply chasing accuracy.
AI-powered Automation
To reduce dependence on manual intervention and technical understanding, the solution should use AI-based automation to reduce the burden on category managers and simplify the decision-making process.
Scalability
A retail solution is fed large sets of data daily. As data continues to evolve in terms of volume, variety, and velocity, the complexity is here to stay. Therefore, your demand forecasting solution must be scalable to ingest and transform data and use AI/ML and other advanced analytics to provide actionable insights.
Easy to Use
Category managers and supply chain planners might not necessarily be tech-savvy. However, they should be able to instantly use advanced features and forecast demand, and react to grocery situations without depending on data scientists and analysts.
If the demand planners require extensive training on the demand forecasting solution to deliver value, they will revert to unproductive behaviors and your returns on the technological investments will be extremely low.
Future-proof Your Retail Forecasting with Algonomy
Your demand forecasting framework must be intelligent, agile, and scalable to efficiently deal with forecasting challenges. Algonomy’s Forecast Right is a one-stop solution for the discussed challenges and more.
Forecast Right leverages ML-based multivariate and algorithmic techniques to accurately and adaptively forecast future demand. Its advanced AI capabilities offer feature engineering and model selection for demand forecasting, with a proven track record of improving forecast accuracy for over 90% of SKUs. Given the inherent benefits of Forecast Right, its output can be further plugged into Order Right for efficient replenishment planning.
Order Right generates accurate SKU level order plans with its proprietary optimization algorithms that account for key supply chain factors such as lead-time, minimum order quantity, etc. and category factors such as shelf-life, expiration date, etc., while constantly monitoring stock balance, sales, and demand predictions.
Request a demo with our experts to discover how Algonomy can help you transform your demand forecasting
