Not all items are created equal. Not all customers are not created equal. We know this to be self-evident. If this is true, why do we assume our accuracy should be the same for all items and customers?

As Demand Planners, it’s not enough to just forecast; we need to understand the underlining forecastability for each item. Forecastability may not be in Webster’s dictionary, but it should be a word we are all familiar with in demand planning. It’s how we measure the underlying uncertainty of a particular item so we can know what kind of accuracy we can expect from it, and what resources are worth allocating to a particular item or category.

To get some insight on this important topic, I invited Sujit Singh, CEO of Arkieva, to discuss this topic on the IBF On Demand Podcast. The following is based on that conversation.

Are All Items Forecastable?

If something has sold only 1 time, then we can agree it is impossible to forecast. Similarly, items with a long tail are not always possible to forecast. Items whose demand is impacted by external variables may not be forecastable either.

Any item that is stable with sufficient data points is forecastable; we can apply a range of techniques to generate a forecast, and in that sense it is forecastable. But the proof of the pudding is in the accuracy – regardless of the data points available or the techniques used to generate the forecast, if the resulting accuracy isn’t sufficient to help plan the business, it is not forecastable.

Items Are Getting Less Forecastable

We have all heard about long tail demand where demand is getting divided into more and more products whereby the portion of demand that is fundamentally unforecastable has increased. Sujit says that with this in mind, we can expand the definition of ‘forecastable’ by generating a forecast range (instead of a point forecast) and as long as we’re inside the range, the forecast is ‘accurate’ and therefore forecastable. In so doing, we can make these tricky-to-forecast long tail items more forecastable.

Reducing Forecast Error During The Pandemic

Given the current demand disruption caused by COVID-19, forecast accuracy is inevitably lower than that we which might have enjoyed prior to the pandemic. Should we increase our tolerance for forecast error? Sujit says if we give more weight to recent observations, isolate certain history and identify certain factors impacting demand, we can still get decent outputs from our time series. Of course, when demand assumptions change, forecast engines aren’t aware. A forecasting system doesn’t know a plant closed down, but your sales team will. With the right information we can then update the models and maintain some degree of accuracy.

Methodologies To Determine If Items Are Forecastable

Error is the main metric to identify which of your times are forecastable, but Sujit recommends another simple (yet useful) metric – Coefficient of Variation. The idea is we calculate the standard deviation of a time series and divide by the mean. Very often, the cutoff point forecasters use for forecastability is a CoV of 0.5 (the lower the number, the better). It’s effective but not a perfect measure of forecastability.

We can also use intermittence i.e., gaps between observations. Let’s say we are looking at data in monthly buckets with a sale in month 1, nothing in months 2 and 3, and a sale in month 4. Calculating the average delta between the non-0 sales. If you are more than 1.2, your series is considered highly intermittent and therefore difficult to forecast. You could still reach some forecast accuracy using specialized methods like Croston’s model, but it’s a challenge.

What Do We Do With Problematic Tail Items?

We all have items we struggle to forecast and would rather forget about. In such cases, once we’ve exhausted other methodologies like getting qualitative inputs from sales, Sujit recommends grouping these items and forecasting at higher levels of aggregation so the individual items ‘inherit’ some of the properties from the top level, thereby making them more forecastable. Let’s say you have 7 products, each of which are unforecastable individually and share some commonalities. Forecast at the group level then disaggregate, applying the weights to each item.

“30% forecast accuracy? Seriously? What do I pay you for?  I could flip a coin and get better results than this!” Yes, we hear this as demand planners. And yes, it hurts – deeply, personally, unjustly.

It’s one of the most frustrating and demoralizing feelings as a Demand Planner to know that you’re trying your darnedest to improve the accuracy of something which you know is largely unforecastable. You’ve maxed-out modeling and model-tuning and have resorted to fishing for any judgemental recommendations that you can get your hands on. The latter likely only helping to see accuracy further struggle with snowballing bias and negative FVA of compounding “expert” overrides. Having thrown everything but the kitchen sink at the problem, you shift your efforts to building-up verbal defences with a Pixar-worthy storyboard of empathy-seeking data challenges and culpability-shifting anecdotes. If only there was a way we could prove to management that this product is not actually forecastable….

We Need To Set Management’s Expectations About What Is Forecastable

Management may be aware of standard forecast metrics – many are introduced in MBA programs, SCOR and Operations textbooks. But typically executives are more worried about how quickly they can show improvement in these measurements than how they are calculated.

Similarly, as demand planners, we are trained and certified on the most common algorithms, performance measurement computations, and off-the-shelf forecast modeling data structure requirements. If we are to set expectations about what is actually forecastable, and what we can actually achieve as demand planners, we need to look beyond these basics. If we don’t we will forever be taking unfair criticism for things outside of our control. What we need to do is not only present our forecast accuracy, but present it alongside forecastability. Forecastability reveals the extent to which an SKU can be forecasted, and provides the crucial context for our forecast accuracy.

Forecast accuracy depends on how forecastable the product is.

Questions To Ask To Gauge An SKU’s Forecastability

What change in forecast accuracy is realized when the best-fit model is recalculated from different assortments of time series horizons?

Are the changes more prevalent for certain model types (hint – they should be for some, especially for more factor-inclusive model types like exponential smoothing)?

What differences in forecast accuracy are observed in monthly, bi-monthly, and quarterly period bucketing?  (Is poor forecast accuracy at the monthly level dramatically improved if consumption and forecast accuracy are looked at in quarterly buckets instead?)

Are any SKU-to-SKU, product line to product line, and product family to product family correlations observed when regression comparisons are run to look for like patterns in the demand history?  Are any of these like patterns accounted for in existing planning bills or bills of materials?

What record counts and financial weighting do the products and model types comprise when categorized into basic segmentation schemas (high value, volatile; high-value, stable; low-value, volatile; low-value stable)?

What are the historic forecastability ranges, within each segments and per product families? (Note: Segmentation can be combined with ABC and Pareto analyses, as well as calculated for markets, customers, or for products within each market/customer.)

Within low-value, volatile records, is inherent demand variability such that the cost of error is more prohibitive than a simple order policy (ex. reorder point or make-to-order)?

By asking these questions, we gain an insight into forecastability – what can be forecasted accurately, and what cannot. We will be able to go the S&OP executive meeting, or sit down with Sales and Marketing, Finance or senior executives, and be able to say with confidence that for a particular SKU, 30% forecast accuracy is a good thing. We can explain why an SKU cannot be accurately forecasted and then make suggestions based on that – after all, knowing that demand for a product cannot be predicted has serious implications for the business. Knowing this allows us as demand planners to mitigate risk and propose the best course of action.

What’s more taking the time to understand what one can forecast and what one cannot, and what results can be expected, one can set better expectations and understanding upfront.

How To Prove That 30% Forecast Accuracy Is A Good Thing

In the opening example, proving that 30% is actually a job well done given the forecastability is important (try re-calibrating your BI tools to show forecast accuracy in terms of variance to CoV). Go one better by showing that the cost avoidance of whatever initiative is actually outweighed by the cost of forecasting for it. If you communicate that the ‘juice-isn’t-worth-the-squeeze’ you can get work off of your plate, allowing you to focus on what matters.

Variability can be lessened by extending the time buckets planned for (daily to weekly to monthly to quarterly to semi-annually), but it is more costly. In certain markets and in certain products, this may be the only option and one that demand planners should constantly be evaluating and influencing. On the flipside is also finding ways to try to improve forecastability by forcing the square peg to better fit into that round hole.

For example, CoV movement over time can be tracked. Where increasing, investigations can be conducted to identify with Commercial colleagues the causes and then the script flipped to challenge what can be done to shape back. Analyzing positive correlating 4P’s effects in more stable products can sometimes yield a playbook to try for your more volatile areas.

Parting Thoughts On Forecastability 

This friendly neighborhood forecaster’s closing reminder is this: You measure how something is setup to execute, and you measure to control or to improve. But if the setup is wrong for the metric, or the metric is wrong for the setup, then you’re allowing the box that you’re in to dictate your success. Break out of the box, see if you need to redesign the box or redesign the metric. In forecasting, one size does not fit all – don’t let spinning on the wheel stop you from asking the question “why?” more often. That is how we understand what is actually forecastable and what isn’t, how to get the credit we deserve, and push the discipline forward.

Stay inquisitive, my friends. That’s the mark of the best Demand Planning professional.

The primary driver for the forecast initiatives was to maximize the return on inventory investment. However, in 2006, FNC started to see diminishing returns on these activities. The cost was exceeding the benefits or the improvements were so small they didn’t create measurable improvements in service levels or inventory turns. At this time, FNC began to think in terms of demand management rather than forecasting. Demand management is a three legged stool consisting of removing non-value added activities, reducing demand variability, and increasing operational flexibility. Our approach was to segment demand planning activities by value and forecastability.

The general purpose of segmenting planning strategies is to mitigate risk. At Fujitsu, we segment the demand planning strategies so that we can apply the appropriate inventory management scheme to each part based on its value and variance characteristics. On average, we have realized a 30% reduction in inventory cost per product while maintaining or improving service levels. These improvements have also created additional savings for our customers in terms of inventory cost and improved service levels. These savings are extremely important in a commodity market where we need to compete on service and survive on thin margins.

Have you had any success with such strategies?  Your comments and questions are welcome.

Barry Chapman
Business Product Manager – Demand Management
Fujitsu Network Communications, Inc.

Hear Barry speak on demand planning segmentation strategies at IBF’s Business Planning & Forecasting: Best Practices Conference in Orlando Florida, November 4-6, 2013