Without KPIs, it is impossible to improve forecast accuracy. Here are 8 highly effective metrics that allow you to track your forecast performance, complete with their formulas.

Forecast Accuracy

This KPI is absolutely critical because the more accurate your forecasts, the more profit the company makes and the lower your operational costs. We choose a particular forecasting method because we think it will work reasonably well and generate promising forecasts but we must expect that there will be error in our forecasts. This error is a function of the time difference between the actual value (Dt) and the forecast value (Ft) for that period. It is measured as:

 Forecast Accuracy: 1 – [ABS (Dt – Ft) / Dt]

Where,

Dt: The actual observation or sales for period t

Ft: The forecast for period t

Our focus on this KPI is to provide insights about forecasting accuracy benchmarks for groups of SKUs rather than identifying the most appropriate forecasting methods. For example, achieving 70-80% forecast accuracy for a newly-launched and promotion-driven product would be a good considering we have no sales history to work from.

SKUs with medium forecastability (volatile, seasonal, and fast-moving SKUs) are not easy to forecast owing to seasonal factors like holidays and uncontrollable factors like weather and competitors’ promotions etc., their benchmark is not recommended to be less than 90-95%.

Tracking Signals

Tracking signals (TS) quantify bias in a forecast and help demand planners to understand whether the forecasting model works well or not. TS in each period is calculated:

 TS: (Dt- Ft) / ABS (Dt – Ft)

Where,

Dt: The actual observation or sales for period t

Ft: The forecast for period t

Once it is calculated, for each period, the numbers are added to calculate the overall TS. When a forecast, for instance, is generated by considering the last 24 observations, a forecast history totally void of bias will return a value of zero. The worst possible result would return either +24 (under-forecast) or -24 (over-forecast). Generally speaking such a forecast history returning a value greater than (+ 4.5) or less than (-4.5) would be considered out of control. Therefore, without considering the forecastability of SKUs, the benchmark of TS needs to be between (-4.5) and (4.5).

Bias

Bias, also known as Mean Forecast Error, is the tendency for forecast error to be persistent in one direction. The quickest way of improving forecast accuracy is to track bias. If the bias of the forecasting method is zero, it means that there is an absence of bias. Negative bias values reveal a tendency to over-forecast while positive values indicate a tendency to under-forecast. Over the period of 24 observations, if bias is greater than four (+4), forecast is considered to be biased towards under-forecasting. Likewise, if bias is less than minus four (- 4), it can be said that the forecast is biased towards over-forecasting. In the end, the aim of the planner is to minimize bias. The formula is as follows:

Bias:  [∑ (Dt – Ft)] / n

Where,

Dt: The actual observation or sales for period t

Ft: The forecast for period t

n: The number of forecast errors

Forecaster bias appears when forecast error is in one direction for all items, i.e they are consistently over- or under-forecasted. It is a subjective bias due to people to building unnecessary forecast safeguards like increasing the forecast to match sales targets or division goals.

By considering the forecastability level of SKUs, the bias of low forecastability SKUs bias can be between (-30) and (30). When it comes to medium forecastability SKUs, since their accuracy is expected to be between 90-95%, bias should not be less than (-10) nor greater than (+10). Regarding high forecastability SKUs, due to their moderate contribution to the total, bias is not expected to be less than (-20) or greater than (20). The less bias there is in a forecast, the better the forecast accuracy, which allows us to reduce inventory levels.

Mean Absolute Deviation (MAD)

MAD is a KPI that measures forecast accuracy by averaging the magnitudes of the forecast errors. It uses the absolute values of the forecast errors in order to avoid positive and negative values cancelling out when added up together. Its formula is as follows:

MAD: ∑ |Et| / n

Where,

Et: the forecast error for period t

n: The number of forecast errors

MAD does not have specific benchmark criteria to check the accuracy, but the smaller the MAD value, the higher the forecast accuracy. Comparing the MAD values of different forecasting methods reveals which method is most accurate.

Mean Square Error (MSE)

MSE evaluates forecast performance by averaging the squares of the forecast errors, removing all negative terms before the values are added up. The squares of the errors achieves the same outcome because we use the absolute values of the errors, as the square of a number will always result in a non-negative value. Its formula is as follows:

MSE: ∑(Et)² / n

Where,

Et: forecast error for period t

n: the number of forecast errors

Similar to MAD, MSE does not have a specific benchmark to check accuracy but the smaller value of MSE, the better forecast model, which means more accurate forecasts. The advantage of MSE is that it squares forecast errors, giving more weight to large forecast errors.

Mean Absolute Percentage Error (MAPE)

MAPE is expressed as a percentage of relative error. MAPE expresses each forecast error (Et) value as a % of the corresponding actual observation (Dt). Its formula is as follows:

MAPE: ∑ |Et / Dt |/n * 100

Where,

Dt: Actual observation or sales for period t

Et: the forecast error for period t

n: the number of forecast errors

Since the result of MAPE is expressed as a percentage, it is understood much more easily compared to other techniques. The advantage of MAPE is that it relates each forecast error to its actual observation. However, series that have a very high MAPE may distort the average MAPE. To avoid this problem, SMAPE is offered which is addressed below.

Symmetrical Mean Absolute Percentage Error (SMAPE)

SMAPE is an alternative to MAPE when having zero and near-zero observations. Low volume observations mostly cause high error rates and skew the overall error rate, which can be misleading. To address this problem, SMAPE come in handy. SMAPE has a lower bound of 0% and an upper bound of 200%. It does not treat over-forecast and under-forecast equally. Its formula is as follows:

SMAPE: 2/n * ∑ | (Ft – Dt) / (Ft + Dt)|

Where,

Dt: Actual observation or sales for period t

Ft: the forecast for period t

n: the number of forecast errors

Similar to other models, there is no specific benchmark criteria for SMAPE. The lower the SMAPE value, the more accurate the forecast.

Weighted Mean Absolute Percentage Error (WMAPE)

WMAPE is the improved version of MAPE. Whilst MAPE is a volume-weighted technique, WMAPE is more value-weighted. When generating forecasts for high value items at the category, brand, or business level, MAPE cancels plus and minus values. WMAPE, however, weights both forecast errors and actual observations (sales). When considered at the brand level, high value items will influence overall error because they are highly correlated with safety stock requirements and development of safety stock strategies. Its formula is as follows:

WMAPE: ∑(|Dt-Ft|) / ∑(Dt)

Where,

Dt: The actual observation for period t

Ft: the forecast for period t

Like other techniques, WMAPE does not have any specific benchmark. The smaller the WMAPE value, the more reliable the forecast.

Alignment of demand and supply has been the subject of extensive research but is still a pain point for many organizations, causing either lost sales on the one hand or holding excess inventory on the other. Unfortunately, under or overstocking is often viewed as binary choice that has to be made, but there is another solution – balancing supply and demand.

Let’s take a look at what under and overstocking means for different functions in the business.

From the point of view of Sales, understocking means:

• Missing sales targets
• Not being able to earn bonuses
• Empty shelves at retailer store meaning lost sales
• Risk of paying penalties to contractual retailers
• Poor customer service which can cause customers to go elsewhere

From the point of view of Supply Chain, overstocking means:

• Limited space in warehouse, causing higher inventory holding costs
• Increased cost of rent if space is not enough to hold stock
• Increased risk of product obsolesce if shelf-life is limited
• Having to offer discounts to clear excess stock, impacting profitability
• Higher labor costs to manage the stock on a regular basis.

Thus, none of the managers on the supply side want to be overstocked but salespeople do so they can take advantage of all opportunities in the market. But it doesn’t have to be either/or. Instead we can find a balance that satisfies the need to both sell as much as possible without incurring the costs of holding excess stock.

How To Find The Balance Between Over & Understocking

1 -Understand Consumer Demand

The first thing is to understand demand, i.e., what consumers want and where. To do so, companies need to learn what shoppers can afford, what products they prefer and why, and environmental and cultural factors that have an impact on consumer behavior.

For instance, if the consumer demands high-end premium products in your store or region, there is no reason to overstock brands that are cheaply made or packaged in boxes with unreadable labels. This calls for historical data to ensure that sales trends, seasonality, and validity in the market can be scanned periodically. With statistical modeling we can take this sales data and extrapolate future demand. Depending on the industry, companies need to review historical sales and update forecasts daily, weekly, monthly and quarterly, depending on the product type.

This approach is also same for suppliers who need to analyze the demand coming from each retailer and store and consumer characteristics. Overall, these actions enable a much clearer picture of what consumers want and what they don’t. When we know this, we have a foundation to start meeting this demand with the necessary supply.

2 – Invest In Your Demand/Supply Planners

A good understanding of demand cannot be achieved with historical data alone. You need good demand and/or supply planners, who are knowledgeable about product groups and categories and aware of external factors that may affect consumption, and who are equipped with knowledge of demand management and forecasting methodologies. Not everything cannot be found in the data. The impact of a competitor’s in-store promotion, cultural impacts forming shopping habits, background of expatriates in the city/region etc. are only some of the factors that planners should consider when generating forecasts, planning supply, and setting stock levels.

Demand management is a specific area that includes many techniques, methodologies and nuances unique to the role, thus making education and training crucial. Planners should know which forecasting techniques to use for which data set, how to aggregate forecasts with factors affecting demand, and when to adjust forecasts with qualitative judgment. Being knowledgeable about a particular product category is a competitive advantage for planners because it allows us to understand the likely impact of promotions and competitor activities. It also enables better communication with the sales team, who we rely on for input into the forecasts and customer information.

3 – Forecasts Feed The Supply Plan

Let me ask you the following question: Does your company look to just hit monthly sales targets or to enhance profitability in the long-run?

If the goal is to just close the month with sales targets achieved, let the sales team create and approve the forecast. This is how it works at most suppliers and distributors. Don’t get me wrong, the contribution of the sales team to any business is incredibly important but they do not have the expertise to create forecasts that represent true demand. Salespeople have the unconscious habit of being optimistic on sales targets, which must be tempered by data-focused demand planners.

When generating forecasts we need input from our colleagues in Sales and Marketing, Finance, Supply Chain, and perhaps Customer Service, and the optimal way to collaborate is through a Sales and Operations Planning process. This is the forum that allows us to align on aggregated forecast numbers. Supply Chain plays a key role here. Let’s take the example of the Tesco, the biggest retail chain in the UK: when Tesco handed the responsibility of order replenishment to their Supply Chain directors, it dramatically increased product availability and reduced inventory. This approach enabled both Supply Chain and category management teams to manage shelf space, promotions and new launch item in a more efficient way.

4 – Integrate Pareto Analysis Into Your Target Stock Level

Pareto analysis, also known as the 80/20 rule, is a statistical method used for decision making which identifies which 20% of inputs leads to 80% of the desired output. In demand planning, we’re looking to identify the 20% of products that contribute to 80% of profitability.

Pareto analysis should be the best friend of planners when it comes to managing inventory. To illustrate further, when I was working for Transmed Overseas, a full service distributor in the Middle East and Africa, we had one single number of DOS (days of stock) target per brand. This was causing massive fluctuations in stock levels and was triggering not only out of stock (OOS) but also excess inventory and obsolete stock. With Pareto analysis, we first categorized the SKUs of each brand from best to worst performing. Then we categorized SKUs that generated 75% of sales as class A and the SKUs that generated 15% of sales as class B. The final group, C, were the SKUs that generated only 5% of sales. This approach identified our most important products and the safety stock levels for each product were determined according to their category.

Using Pareto analysis, we not only reduced excess stock by 15 to 20%, but also ensured availability of class A SKUs, which improved customer service by 3%. As valuable as Pareto analysis, is, you must also consider lead time, contractual agreements, forecast accuracy, and other factors.

5 – Optimize Order & Replenishment Frequency

If we get our inventory replenishment frequency right, we reap the rewards of lower inventory. It is easy to write but difficult to apply! Of course, there are many factors affecting the right order frequency such as long-lead times, seasonality, forecast accuracy, containerization, promotions, and PIPO (Phase in Phase out) practices, but it is doable.

Your starting point is to check whether your lead times are accurate or not. Without a high level of lead time accuracy, any attempt to increase order frequency is shot in the dark that risks failure and cost. Thus, the supply chain team should work meticulously to track OTIF (On time, in full) performance of every purchase from each supplier. Once there is a reliable history on lead time with accuracy performance, then the team should check forecast accuracy at the SKU level and forecast misses, which is one of the invisible inventory costs incurred by companies. Improvement in lead time and forecast accuracy will increase the confidence in replenishing products on time and in full at DCs and stores. Following this, containerization should be analyzed which has a direct impact on logistics and transportation costs. This responsibility lies with the Supply Chain team, who should compare the cost of inventory holding, forecast misses, and obsolesce versus savings from logistics and transportation. This cost/savings ratio should inform your ordering frequency.

Amending your order frequency should consider marketing or category management teams because they run promotions that can impact the amount of inventory you need. If promotions are not factored into the lead times and not communicated to the Supply Chain and Procurement teams, plans will not include the promotional volume. This not only gives rise to missed sales/understocking, but also poor customer service, and even penalties at the downstream level depending on your agreements with retailers.