I cannot discuss forecasting bias without mentioning MAPE, but since I have written about those topics in the past, in this post, I will concentrate on Forecast Bias and the Forecast Bias Formula.
[bar group=”content”]

What Is Forecast Bias?

Forecast Bias can be described as a tendency to either over-forecast (forecast is more than the actual), or under-forecast (forecast is less than the actual), leading to a forecasting error.

There are many reasons why such bias exists including systemic ones as discussed in a prior forecasting bias discussion. Some core reasons for a forecast bias includes:

1. Optimism bias: I have seen this primarily with the sales team who seem to have an abundance of confidence in their ability to sell and therefore inflate the end results.
2. Sandbagging bias: This is the reverse of the above and I have seen this where well-meaning executive have created a system of bonuses based on exceeding the forecasts, and this has created a culture of sandbagging.
3. Anecdote bias: I have heard so many instances where regardless of what the data is telling them, client personnel would be wary of seeing it because a terrible thing that happened in the past and is part of the company folklore. Their forecast is therefore biased based on the anecdotes.
4. Recent data bias: This is probably true for all processes where humans are involved. The more recent occurrences weigh heavier in our mind. In the case of forecasting, this can create an overreaction based on the latest events.
5. Silly bias: In a study conducted by Amor Tversky and Daniel Kahneman, they asked respondents to guess the number of countries in Africa. However, they showed them a number right before asking them to guess. What they found was on average, the estimate of some countries went up when the user was shown a bigger number and went down when the users were shown a smaller number before answering the question. This makes me think a forecast could be impacted by silly things you saw before you start doing the forecast. For example, what if they saw the temperature and it was a hot day? Does that high number skew the forecast higher? What if they called someone before forecasting and the phone number was comprised of larger digits?

How To Calculate Forecast Bias

A quick word on improving the forecast accuracy in the presence of bias. Once bias has been identified, correcting the forecast error is quite simple. It can be achieved by adjusting the forecast in question by the appropriate amount in the appropriate direction, i.e., increase it in the case of under-forecast bias, and decrease it in the case of over-forecast bias.

Rick Glover on LinkedIn described his calculation of BIAS this way: Calculate the BIAS at the lowest level (for example, by product, by location) as follows:

• BIAS = Historical Forecast Units (Two-months frozen) minus Actual Demand Units.
• If the forecast is greater than actual demand than the bias is positive (indicates over-forecast). The inverse, of course, results in a negative bias (indicates under-forecast).
• On an aggregate level, per group or category, the +/- are netted out revealing the overall bias.

The other common metric used to measure forecast accuracy is the tracking signal. On LinkedIn, I asked John Ballantyne how he calculates this metric. Here was his response (I have paraphrased it some):

• The “Tracking Signal” quantifies “Bias” in a forecast. No product can be planned from a severely biased forecast. Tracking Signal is the gateway test for evaluating forecast accuracy. The tracking signal in each period is calculated as follows:

• Once this is calculated, for each period, the numbers are added to calculate the overall tracking signal. A forecast history entirely void of bias will return a value of zero, with 12 observations, the worst possible result would return either +12 (under-forecast) or -12 (over-forecast). Such a forecast history returning a value greater than 4.5 or less than negative 4.5 would be considered out of control.

At Arkieva, we use the Normalized Forecast Metric to measure the bias. The formula is very simple.

As can be seen, this metric will stay between -1 and 1, with 0 indicating the absence of bias. Consistent negative values indicate a tendency to under-forecast whereas constant positive values indicate a tendency to over-forecast. Over a 12-period window, if the added values are more than 2, we consider the forecast to be biased towards over-forecast. Likewise, if the added values are less than -2, we find the forecast to be biased towards under-forecast.

A forecasting process with a bias will eventually get off-rails unless steps are taken to correct the course from time to time. A better course of action is to measure and then correct for the bias routinely. This is irrespective of which formula one decides to use.

Good supply chain planners are very aware of these biases and use techniques such as triangulation to prevent them. Eliminating bias can be a good and simple step in the long journey to an excellent supply chain.

It is essential to continuously revisit and modernize your demand forecasting processes so they adapt to current market realities. This article outlines the basic yet often overlooked steps that need to be followed to ensure that your forecasting process reflects the latest operational setup, provides value with minimal waste and, most importantly, enhances your business’s bottom line.

1. Forecast Keys: What is The Right level To Forecast At?

Determination of the forecast keys ultimately needs to be tied to the organizational processes that demand forecasting serves. Historically, demand forecasting has been established along the product, location and period dimensions. It is important to evaluate multiple options that will achieve greater forecast accuracy:

• You might find that you can produce more accurate results when generating forecasts at the lowest level by having your model capture true consumer demand signals. In this case you can roll such forecasts up to the needed level.
• On the other side, due to sparsity of data, you might learn that forecasting at the higher level is a better option. In this case, to serve your downstream purposes, you can spread the data to the required level.

It is essential to revisit these decisions on a periodic basis. With marketing incentives being tied to CRM outputs, customer segment might be an important dimension for demand forecasting. In an omnichannel setting, you might find that extending your location dimension to geographic area might add more value. Therefore, continue to assess demand shaping activities and adjust your forecasting process accordingly.

2. Demand Drivers: What Inputs To Plan For?

Demand forecasting is becoming more sophisticated with Machine Learning algorithms. Machine Learning engines establish forecasting for total demand with a single approach, allowing modeling for direct impacts and interactions between the features. In addition to typical seasonality, holiday traffic and various promo incentives modeling, demand forecasting can also be enhanced by incorporating additional data feeds like demographics, weather, product images, etc.

Leveraging the latest algorithms is the path forward, however such conversion requires organizational effort despite the convenience of mining data externally. Forecasting output is only as good as the input data. It should be a coordinated effort across multiple teams within the organization including the product, placement, pricing, promotions and planning teams. Their input allows you to continuously feed good quality data into the forecast engine.

3. Forecasting Frequency And Period: How Often And Long To Forecast For?

The length of a forecast and its frequency are interrelated. It is known that forecasts further into the future are less reliable. Hence, forecasts need to be updated on a periodic basis. It is worthwhile to assess how often your demand driving inputs change and by how much:

• You can find that for some of your products, changes are so frequent and justifiable that you can outweigh the cost of forecasting on a daily or even hourly basis.
• On the other side, you can have another set of intersections in your mix with far more stable behavior where such frequent efforts provide absolutely no value.

Organizations can also exploit forecasting for a range of values rather than a single point to allow processes like ordering to function on the probabilities.

4. Forecast Snapshots: How Far In Advance To Freeze Forecasts?

As forecasts update continuously when moving closer to the forecast period, it is important to establish a cadence around taking forecast snapshots to be used for accuracy evaluation. All demand forecasting outputs used across multiple organizational processes should have their own snapshot. When the forecast frozen horizon is determined, it is too late to change your predicted quantity decision.  

5. Forecast Accuracy: What Forecast Error Measurements To Use?

There are multiple forecast error measurements to use but all of them are essentially based on how far the forecast was from the actual demand. It is known that business impact from underforecasting can result in lost sales leading to customer dissatisfaction, while overforecasting leads to excess inventory costs, spoilage, etc. Yet organizations often blend under and overforecasting when aggregating errors, computing absolutes etc.  You should “punish” your underforecast (negative error of forecast – actual) vs. overforecast (positive error of forecast – actual) separately:

Overforecast Cost*Positive  Error – Underforecast Cost* Negative  Error)/Actual Sales Units

The above is a basic conceptual expression of a formula that could be used for overall error results driven by the individual product & location aspects. With this approach you can better target the bottom line by tweaking your forecasting boundaries to be more tolerant to overforecasting in areas where lost sales are more damaging and favor underforecasting output when excess inventory cost and spoilage are not that affordable.

6. Forecast Accuracy Roll-ups: How To Action On Overall Results?

Organizations tend to measure forecast accuracy at the aggregate levels, while more actionable results are identified in the lower level mix.  It is not helpful to know that out of the entire volume of products forecasted, these were matching to the actual sales units, but rather focus on making sure that the right products were available in the right locations at the right time.

When looking into the aggregated and properly weighted results, one can then do investigation by drilling down to the top worst offenders of high errors. This can lead to action items for reducing errors in areas having the highest financial impact.

7. Forecast Benchmarking: What Is The Right Yardstick To Compare Forecast Accuracy To?

For being one of the most common questions, unfortunately there is no one simple answer, as it really depends on the nature of the intersections needed to be forecasted. It is known that forecast results tend to be more accurate for higher sales volume and more stable demand, rather than sparse and very volatile intersections.

You should be setting different targets depending on forecastability, or ease of predicting accurate results depending on the historical behavior.

8. Forecast Streamlining: How To Focus Resources On The Value Add Areas?

It is recommended to classify forecasted intersections into importance/error-variation groups. Depending on organization size and assortment, you can establish as few as 4 quadrants, relying on integrated ABC-XYZ analysis, where ABC can represent contribution from margin, volume or other important characteristics and XYZ would be classified by historical accuracy, demand variability and related forecastability characteristics.

You can allow your forecast engine to produce forecasts automatically across all the intersections and have your resources to focus on more value adding activities.  

9. Forecast Impact: Why Do End To End Simulations Matter?

In perfecting forecasting outputs, it is sometimes easy to get trapped into wasteful analysis. By analyzing your data, you might learn that for most of the very slow moving products, forecast values have very little impact and real effect is driven by other triggers. Hence, it always worthwhile to perform end to end simulation to understand the boundaries and associated thresholds beyond which intensification of forecasting efforts is wasteful.

10. Forecast Exceptions: When Is Automation Not Enough?

With all the power of forecasting algorithms feeding from multiple data sources, it is still important to establish a process for exceptions. Basic rules and sanity checks of what do not make sense based on historical data, typical user judgement and common observations would form a base for such exceptions. Center your exceptions around important activities, define your tolerance based on financial impact and actionability of a given exception. Having an exceptions mechanism in place serves as a system of checks and balances to make your forecasting a well-rounded process that can function automatically, yet identify the focus areas and alert the needed personnel.

Can you forecast better than a first grader? I would hope so, and of course most of us can. After all, our experience with forecasting tools and methodology far exceed those of children, our brains bigger ‘s, and we have more life  experience to help us make informed judgements. Or none of that is actually true, and all these years of accumulated knowledge actually puts you at a disadvantage to an 8 year old.

Kids live by their instincts, openly and without hesitation. They are enthusiastic about life, eager to learn, and curious about everything. Even though kids still have a lot to learn about predictive modeling using reinforcement learning techniques, just observing them can teach us many other practical lessons. These lessons are simple, yet as we focus on analytics, data, and the daily grind of our adult occupation, we may overlook what we could learn from our younger selves.

Here are six characteristics kids and preschool problem solvers have that we may want to think about emulating to improve our predictive analytics abilities.

1  – Children Don’t Make Assumptions

Children are more exploratory and more likely to change their minds, considering a wider range of possibilities – including even those that are unlikely. Adults are more hesitant to revise their beliefs. And even when they do so, they may only consider alternatives that they believe are likely to be true. In doing so, we walk away from numerous possibilities especially the unlikely.

Probabilistic thinking depends on the balance between “priors” – the beliefs we bring to a problem, and data. As we get older our “priors,” rationally enough, get stronger and stronger. We rely more on what we already know, or think we know, and less on new data.  Maybe we should approach data through the eyes of a child more and instead of causal relationships between variables, go on more fishing exercises for any possible relationship between any variables.

Children examine the details of setbacks though curious eyes and so they become learning experiences.

2- They Can be Wrong with Confidence

Kids are rarely mired by hesitation or fear.  They usually dive head first into every situation, leaping over hurdles and improvise their approach when they must. As we get older, we begin to over-analyze our next move and hesitate because we’re afriad that we may not get it right. Most adults solve problems by wracking their brains for an existing solution and giving up if they can’t identify one. Children solve problems more creatively; they use trial and error when reference isn’t available. As adults – in our field especially – we must deal with ambiguity daily. There is something to say for diving in head first and being wrong with confidence.

The unbiased curiosity of children provides insights into how we can be better forecasters

3- They Laugh and Keep Going

When kids are dealt an unexpected hand, they usually laugh it off and work around it. Kids just don’t seem to wallow in the things out of their control the way adults do. As adults, we become good at bringing up problems but spend little time on solutions. We see obstacles as insurmountable impediments, and setbacks as failures.  Instead, children examine the details of the outcome though curious eyes and so they become learning experiences. Remember this the next time you are reviewing forecast performance and instead start using setbacks to understand and learn more about variability and the process, rather than as a measure of individual performance.

Adults accept the status quo for what it is and rarely  search for better solutions.

4 -Kids Ask Lots of Questions

Why? How? Are we there yet?  Kids are curious and if they don’t know they are not afraid to ask someone else.  And more importantly they are not shy about questioning everything. While most adults embrace black-and-white answers to complex questions, those who keep asking “why” become the catalysts of change. Kids challenge their environment and ask why we do things the way we do. Adults accept the status quo for what it is and rarely  search for better solutions. Assumptions, models, almost everything we do or use is dynamic and as soon as we stop asking questions we stop getting answers.

5 – Children Think Everything Is Possible

As far as they are concerned every problem can be solved. What’s more, young children get praise and encouragement from their parents and teachers for almost any work they do. Adults on the other hand are only too well versed in what they cannot achieve and what cannot be done. They have experienced rejection, failure and limitations. Unfortunately, most companies reinforce this type of behavior. As individuals we need to challenge what we thought we knew to accomplish something faster, smarter or better than a current method. As companies, we don’t always want to praise only winners but also those who really put in an effort, or have a constant flow of suggestions. You will find this is the only way to lead to innovations and seeing that anything is possible.

6 – Kids Are Always Learning

Kids observe the people around them with enthusiastic intensity.  They typically mimic the actions that work and ignore the ones that don’t. As we grow older, we tend to become less observant, instead relying solely on formal instruction for expanding our skill set. We forget how well we learned when we were young by simply observing things. This has a couple of implications: to succeed in our volatile, complex, ambiguous world, we have no choice but to master our ability to adapt and keep learning. This is attending conferences, reading and writing articles, and keeping ahead of trends in our field. But learning is also being observant daily in what you are doing. It is seeing small significant patterns in data, uncovering key assumptions that others may overlook, or observing elements that may be contributing bias.

Final Thoughts

Maybe to get back some of the edge we had as kids, we need to start acting more childlike. Perhaps it is OK be a bit more childish and laugh more.

As adults we cannot let what we have learned through years of experience become the bias that stops us from growing. In the fields of Demand Planning, Forecasting, and Predictive Analytics we need to keep an open mind about data, never stop asking questions, challenge assumptions, and push past the limits of what is possible.  And most of all, never stop learning.

If, like me, you work in a small to medium sized enterprise where forecasting is still done with pen and paper, you’d be forgiven for thinking that Machine Learning is the exclusive preserve of big budget corporations. If you thought that, then get ready for a surprise. Not only are advanced data science tools largely accessible to the average user, you can also access them without paying a bean.

If this sounds too good to be true, let me prove it to you with a quick tutorial that will show you just how easy it is to make and deploy a predictive webservice using Microsoft’s Azure Machine Learning (ML) Studio, using real-world (anonymised) data.

What is Azure ML?

To most people the words ‘Microsoft Azure’ conjure up vague ideas of cloud computing and TV adverts with bearded-hipsters working in designer industrial lofts, and yet, in my opinion, the Azure Machine Learning Studio is one of the more powerful and leading predictive modelling tools available on the market. And again, its free.  What’s more, because it has a graphical user interface, you don’t need any advanced coding or mathematical skills to use it. It’s all click and drag. In fact, it is entirely possible to build a machine learning model from beginning to end without typing a single line of code. How’s that for a piece of gold?

You can make a free account or sign in as a guest here – https://studio.azureml.net The free account or guest sign-in to the Microsoft Azure Machine Learning Studio gives you complete access to their easy-to-use drag and drop graphical user interface that allows you to build, test, and deploy predictive analytics solutions.  You don’t need much more.

Microsoft Azure Tutorial Time!

I promised you a quick tutorial on how to make a forecast that drives purchasing and other planning decisions in Azure ML, and a quick tutorial you shall have.

If you’re still with me, here are a couple of resources to help you get rolling:

A great hands on lab: https://github.com/Azure-Readiness/hol-azure-machine-learning

Edx courses you can access for free: https://www.edx.org/course/principles-machine-learning-microsoft-dat203-2x-6

https://www.edx.org/course/data-science-essentials-microsoft-dat203-1x-6

Having pointed you in the direction of more expansive and detailed resources, it’s time to get into this quick demo. Here are the basic steps we’ll go through:

• Uploading datasets
• Exploring and visualising data
• Pre-processing and transforming
• Predictive modelling
• Publishing a model and using it in Excel

Uploading Datasets To Microsoft Azure

So, you’ve signed up. Once you’re in, you’re going to want to upload some data. I’m loading up the weekly sales data of a crystal glass product for the years 2016 and 2017 which I’m going to try and forecast.  You can read in a flat file csv. format by clicking on the ‘Datasets’ icon and clicking the big ‘+ New’:

   Then you’re going to want to load up your data from the file location and give it a name you can find easily later. Clicking on the ‘flask’ icon and hitting the same ‘+ New’ button will open a new experiment. You can drag your uploaded dataset from the ‘my datasets’ list on to the blank workflow:

Exploring and Visualizing

Right clicking on the workflow module number (1) will give you access to exploratory data analysis tools either through ‘Visualise’, or by opening a Jupyter notebook (Jupyter is an open source web application) in which to explore the data in either Python or R code. If you want to learn how to use and apply Python to your forecasting, practical insights will also be revealed at IBF’s upcoming New Orleans conference on Predictive Business Analytics & Forecasting.

Clicking on the ‘Visualise’ option calls up a view of the data, summary statistics and graphs. A quick look at the histogram of sales quantity shows that the data has some very large outliers. I’ll have to do something about those during the transformation step. You also get some handy summary statistics for each feature. Let’s have a look at the sales quantity column.

I’m guessing that zero will be Christmas week, when the office is closed. The max is likely to be a promotional offer. I can also see that the standard deviation is nearly 12,000 pieces, which is high compared to the mean. You can also compare columns/features to each other to see if there is any correlation:

Looking at a scatter plot comparison of sales quantity to the consumer confidence index value, that really doesn’t seem to be adding anything to the data. I’ll want to get rid of that feature. I’ve also included a quick Python line plot of sales over the two-year period.

As you can see, there is a lot of variability in the data and perhaps a slight downward trend. Without some powerful explanatory variables, this is going to be a challenge to accurately forecast. A lot of tutorials use rich datasets which the Machine Learning systems can predict well to give you a glossy version. I wanted to keep this real. I work in an SME and getting even basic sales data is an epic battle involving about fifty lines of code.

Pre-processing and Transforming

Now it’s time to transform the data. For simplicity, I’ve loaded a dataset with no missing or invalid entries by cleaning up and resampling sales by week with Python, but you can use the ‘scrub missing values’ module or execute a Python/R script in the Azure ML workspace to take care of this kind of problem.

In this case, all I need to do is change the ‘week’ column into a datetime feature (it loaded as a string object) and drop that OECD consumer confidence index feature as it wasn’t helping. I could equally have excluded the column without code using the select columns module:

One of the other things I’m going to do is to trim outliers from the dataset using another ‘Execute Python Script’ module to identify and remove outliers from the sales quantity column so the results are not skewed by rare sales events.

Again, I could have accomplished a similar effect by using Azure’s inbuilt ‘Clip Values’ module. You genuinely do not have to be able to write code to use Azure (but it helps.)

There are too many possible options within the transformation step to cover in a single article. I will mention one more important step. You should normalise the data to stop differences in scale of the features leading to certain features dominating over others. 90% of the work in forecasting is getting and cleaning the data so that it is usable for analysis (Adobe, take note. Pdf’s are evil and everyone who works with data hates them.) Luckily, you can do all your wrangling inside the machine model, so that when you use the service, it will do all the wrangling automatically based on your modules and code.

The Normalize data module allows you to select columns and choose a method of normalisation including Zscores and Min-Max.

Predictive Modelling In Microsoft Azure

Having completed the data transformation stage, you’re now ready to move on to the fun part – making a Machine Learning model. The first step is to split the data into a training set and a testing set. This should be a familiar practice for anyone working in forecasting. Before you let your forecast out into the wild you want to test how well it performs against the sales history. It’s that or face a screaming sales manager wanting to know where his stock is. I like my life as stress-free as possible.As with nearly everything in Azure ML, data splitting can be achieved by selecting a module. Just click on the search pane and type in what you want to do. I’m going to split my data 70-30.

The next step is to connect the left output of the ‘Split Data’ module to the right input of a ‘Train Model’ module, the right output of the ‘Split Data’ to a ‘Score Model’ module, and a learning model to the right input of the ‘Train model’.

At first this might seem a little complicated, but as you can see, the left output of the ‘Split Data’ is the training dataset which goes through the training model and then outputs the resulting learned technique to the ‘Score Model’ where this learned function is tested against the testing dataset which comes in through the right data input node. In the ‘Train Model’ module you must select a single column of interest. In this case it is the quantity of product sold that I want to know. 

Microsoft offer a couple of guides to help you choose the right machine learning algorithm. Here’s a broad discussion and if short on time, check this lightning quick guidance. In the above I’ve opted for a simple Linear Regression module and for comparison purposes I’ve included a Decision Forest Regression by adding connectors to the same ‘Split Data’ module. One of the great things about Azure ML is you can very quickly add and compare lots of models during your building and testing phase, and then clear them down before launching your web service.

Azure ML offers a wide array of machine learning algorithms from linear and polynomial regression to powerful adaptive boosted ensemble methods and neural networks. I think the best way to get to know these is to build your own models and try them out. As I have two competing models at work, I’ve added in an ‘Evaluate Model’ module and linked in the two ‘Score Model’ modules so that I can compare the results. I’ve also put in a quick Python script to graph the residuals and plot the forecasts against the results.

Here’s the Decision Forest algorithm predictions against the actual sales quantity:

Clearly something happened around May 2016 that the Decision Forest model is unable to explain, but it seems to do quite well in finding the peaks over the rest of the period 2017. Looking at the Linear Regression model, one can see that it does a better job of finding the peak around May 2016 but is consistently overestimating in the latter half of 2017.

Clicking on the ‘Evaluate Model’ module enables a more detailed statistical view of the comparative accuracy of the two models. The linear regression model is the top row and the decision forest model is the bottom row.

Coefficient of determinations of 0.60 and 0.72. The models are explaining between half and three-quarters of the variance in sales. The Decision Forest overall scored significantly better. As results go, neither brilliant nor terrible. A perfect coefficient of determination of 1 would suggest the model was overfitted and therefore unlikely to perform well on new data. The range of sales was from 0 to nearly 80,000, so I’ll take 4421 pieces of mean absolute error without a complaint.

It would really be ideal if we had a little more information at the feature engineering stage. The ending inventory in-stock value from each week, or customer forecasts from the S&OP process as features would help accuracy.

One of the benefits of forecasting in this way is you can incorporate features without having to worry about how accurate they are as the model will figure that out for you. I’d recommend having as many as possible and then pruning. I think the next step for this model would be to try incorporating inventory and S&OP pipeline customer forecasts as a feature. Building a model is an iterative process and one can and should keep improving it over time.

Publishing A Model And Consuming It In Excel

Azure ML makes setting up a model as a webservice and using it in Excel very easy. To deploy the model, simply click on the ‘Setup Web Service’ icon at the bottom of the screen.

Once you’ve deployed the webservice, you’ll get an API (Application Programming Interface) key and a Request Response URL link. You’ll need these to access your app in Excel and start predicting beyond your training and testing set. Finally, you’re ready to open good old Excel. Go to the ‘Insert tab’ and select the ‘Store’ icon to download the free Azure add-in for Excel.

Then all you need to do is click the ‘+ Add web service’ button and paste in your Response Request URL and your secure API key, so that only your team can access the service.

After that it’s a simple process to input the new sales weeks to be predicted for the item and the known data for other variables (in this case promotions, holiday days in the week, historic average annual/seasonal sales pattern for the category etc.). You can make this easy by clicking on the ‘Use sample data’ to populate the column headers so you don’t have to remember the order of the columns used in the training set.

Congratulations! You now have a basic predictive webservice built for producing forecasts. By adding in additional features to your dataset and retraining and improving the model, you can rapidly build up a business specific forecasting function using Machine Learning that is secure, shareable and scalable.

Good luck!

If you’re keen to leverage Python and R in your forecasting, we also recommend attending IBF’s upcoming Predictive Analytics, Forecasting & Planning conference in New Orleans where attendees will receive hands-on Python training. For practical and step-by-step insight into applying Machine Learning with R for forecasting in your organization, check out IBF’s Demand Planning & Forecasting Bootcamp w/ Hands-On Data Science & Predictive Business Analytics Workshop in Chicago.

The importance of a forecast really depends on the environment in which you operate. If you are fortunate enough to be in an environment where customer orders coincide with your cycle time, then the forecast is really more for mid- to long-range planning. However, if you are in an environment like mine, you promise a three-day ship even though it takes three months to procure the materials. This is where the forecast drives everything you do on a daily basis. To be successful in that kind of environment requires a much higher level of engagement. Here I share what I found to be “The Ten Commandments” of the forecasting profession.

First Commandment: Have Fun In Your Forecasting Role

People can sense whether or not you love what you are doing. People tend to have more confidence in someone who loves what he or she does and takes pride in doing it. Enjoy the fact that in forecasting you can make a living by being wrong all the time. But, if you don’t enjoy crunching numbers or interfacing with a wide variety of personalities, then forecasting is probably not for you.

Salespeople are paid to sell, not to forecast. Therefore, to obtain input from them, we must educate them

Second Commandment Pretend You Know How The Economy Impacts Your Business

It won’t matter whether you are right or wrong. What matters is that you took that element into account and that you sound like you know what you are talking about. Who knows? You may even get lucky and find a leading economic indicator that will help to improve the forecast.

Third Commandment: Be Able To Discuss The Complex Algorithms Without Boring Your Audience

Most people don’t like statistics, so make even the high-level algorithms simpler and easier to understand. With this ability, people will think you actually know what you are talking about.

Fourth Commandment: Don’t Take The Numbers Provided By Sales Or Marketing As Gospel

Take the time to review and validate them against history, and then get out and interact with them. If you disagree with their numbers, talk to them about why you feel your numbers might be better. They are guessing anyway, so they will appreciate any constructive input you may provide, especially if they are being held accountable for the accuracy of their numbers.

Fifth Commandment: Establish A Forecast Accuracy Metric That Is Objective, Quantitative, And Manageable

Whichever metric you choose, it should:

• Accurately depict whether the forecast is getting better or worse

• Point to items that need the most improvement

• Measure accuracy at your procurement lead time (typically three months)

• Provide accuracy information by customer, branch, brand, product category, etc.

• Be automated so that it does not take days to compile that information. I would suggest a weighted forecast accuracy that will allow you to aggregate the data into one forecast accuracy number for any view you choose. It does not matter what the number is; what matters is setting a baseline against which improvement can be measured.

I once told what I thought to be a very funny joke to an Accountant who sat there staring at me as if I had just insulted his entire family

Sixth Commandment: Obtain Input From Different Stakeholders

The next step is to get input from various stakeholders. No one has all the information needed to make a forecast. You need input from Sales & Marketing, and Finance, etc., but always remember, they are not paid to forecast, you are. Salespeople, for example, are paid to sell, not to forecast. Therefore, to obtain input from them, we must educate them. Once they understand that they also have a stake in good forecasts, they will be more willing to contribute.

What is important is that you have a consensus, and everyone agrees on the numbers at both the upper and lower ends

Seventh Commandment: Review With The Participants The Dollar Forecasts At An Aggregate Level Each Month

If the dollars look good, there is no need to get into all the ugly SKU details unless someone has a specific issue to discuss. You should also be prepared to discuss the exceptions. Where were the big misses last month? Are the phase-ins and phase-outs properly forecasted? What promotions are coming up? Get Sales/Marketing to buy into the high-level dollars and then have them spend the rest of the time addressing “events” that the statistical package did not factor in. You can have 10 people looking at the same set of numbers, and all the 10 will come up with a different forecast. No particular one is any more appropriate than the other. This is where the Forecast Analyst rules.

There is no point in wasting anyone’s time reviewing statistically sound forecasts unless there is an upcoming “event” that needs to be considered. At the end of the day, what is important is that you have a forecast consensus, and everyone agrees on the numbers at both the upper and lower ends. You will know you have a consensus when you can go into a meeting and not hear “those numbers aren’t right.” That, more than anything else, will jeopardize the integrity of the entire process as well as the effectiveness of the Forecast Analyst.

Eighth Commandment: Understand Your Audience

You must work with Sales (who would rather be out selling), Marketing (who would rather be picking out all the pretty colors), Finance (who wonders why the one report is $.06 off from the previous report), and Production and Inventory Control (who will be using your forecast in its production planning). With the wide variation in personalities you encounter, you must be flexible and willing to adjust your communication style to your audience. I once told what I thought to be a very funny joke to an Accountant who sat there staring at me as if I had just insulted his entire family. It did not turn out to be the icebreaker I’d hoped for, because I did not adjust my presentation to the audience.

Ninth Commandment Involve P&IC Early On In The Process

Typically, nobody knows more about the product categories and SKUs than Production & Inventory Control associates. They will be the first ones to bring any product mix issues to your attention. Get them involved early on in the process so that the mix can be adjusted before the planning system starts sending out bad messages. Nothing upsets P&IC (Production & Inventory Control) more than coming in on a Monday morning and being told by MRP to make/buy all the wrong parts.

The high-level dollars may look fine, but this is where the rubber meets the road. Make sure you are spending enough time at the SKU level detail. An SKU forecast accuracy metric and exception-based reports will help you sift through the possibly unmanageable number of SKUs you are responsible for forecasting. If you are responsible for a couple of hundred parts, you may very well be able to forecast at a SKU level. But, if you have tens of thousands of parts, then you are probably forecasting at an aggregate level, thereby possibly overlooking the importance of forecasts at the SKU level.

I learned early on in my career that people are much more likely to help an idea succeed if they are part of the design process

Tenth Commandment Keep Your Eyes Open

There are countless webinars, seminars, and books that can help you stay on top of any developments in Demand Planning. IBF is one key resource that I have used over the years, but there are many more that can also help you develop your skills and processes. Don’t stagnate and fall behind what others are doing in the field. New ideas help stimulate other new ideas, which make the job interesting. Most new ideas cannot just be dropped in. You need to use your creativity to adapt them to your organization. This requires a very unique skill set that most people struggle with. This is why it is critical to implement any change as a “team.”

I learned early on in my career that people are much more likely to help an idea succeed if they are part of the design process. If you just try to push it through, I guarantee you will experience a lot of resistance In the end, I would say if you are a Demand Planner/Forecaster or hope to be one someday, stay focused on improving the forecast accuracy while making it fun for yourself and for others. Once the organization has the confidence in the forecast, it will be used to drive both short and long-term business decisions. Forecasts are the key input to any S&OP (Sales & Operations Planning) and/or IBP (Integrated Business Planning) process.

This article first appeared in the Fall 2011 issue of the Journal of Business Forecasting. To receive the Journal and other benefits, sign up for IBF membership today.

Global companies without an integrated global S&OP process are not using their assets in the most efficient way. Or, it can be said that they are not serving their customers with the best service or the most cost-efficient option. This could be due to several reasons: resistance to change, acquisitions, poor communication, silo management or lack of business integration.

Today, multi-national companies are all competing to be more cost-efficient but, in a customer-centric economy, they need to carefully balance the tradeoffs. Supply Chain professionals are the ones called in to solve this problem with innovative and practical solutions. The benefits of global S&OP rollout are:

• Cost reduction
• Better visibility
• Expansion into new territories
• Improve chances of successful new product launch
• Reduced complexity
• Removal of inconsistencies across the network

Buy-in and collaboration are crucial to any change management project; if there’s no collective will to implement it, it is destined to fail.

For a company to successfully roll out a Global S&OP process, it needs 3 core elements:

[bar group=”content”]

1. Helicopter View On Their Regional S&OP

If they manage their regional S&OP teams separately without any global perspective, the company will not be able to leverage them to create a global network – no matter how mature the local S&OP is. A global outlook is needed, as is a similar level of maturity in the local S&OP organizations. Once the company is able to gather and analyze the current global situation, they can then move forward with potential solutions.

2. Total Cost of Ownership (TCO)

Having a global overview of supply and demand means you can optimize the supply chain network (SCN). The TCO plays a key role because it provides the total cost of serving customers from the different facilities that the company owns. The objective will be to minimize the total TCO for the SCN without impacting the customer service level.

3. Change Management Plan

Once the opportunities to optimize the global SCN are identified, the stakeholders will need to come up with a project to catch those opportunities. A change management plan is essential to highlight the importance of the project, align the different regions and stakeholders, and to avoid power struggles. Buy-in and collaboration are crucial to any change management project; if there’s no collective will to implement it, it is destined to fail.

This is not to mention the importance of technology, people and processes. Integration of regional S&OP organization requires software and the people to use it, and structured processes implemented by visionary leaders who may or may not currently exist in your company. Further insight into planning and implementing global S&OP is featured in this edition of the Journal of Business Forecasting.

In November 2017, Matias spoke at ‘Business Planning, Forecasting and S&OP: Europe’ in Amsterdam, alongside Nestlé, Diageo, Johnson & Johnson, Heineken and more. We look forward to welcoming you to next year’s conference in Amsterdam, on 14-16 November 2018.

If you have any knowledge of Demand Planning, I am sure that you have heard the following: “Demand Planners are like meteorologists, they rarely get credit for doing the job correctly and they’re only noticed when they get it wrong.” Even so, the bottom line is that there are serious and costly ramifications which can occur if these decisions are wrong. For this reason, the demand planning position can be one of the most important and visible in the company. It is a great place to impact many areas of business and gain corporate approval. It is best to take a positive approach and be an agent for fact based decision making. Using this approach, along with good communication skills, is a great avenue to gain the knowledge and build relationships that will prepare you for success and lead you along a career path with much variety.

Demand Planning is Transferable To Any Industry

Demand Planning touches every aspect of the business and the impact can make this person a valuable asset very quickly. It requires broad business knowledge and detailed customer interaction. Also, it is a functional area that has the ability to transfer these skills to any industry. It involves working with several areas of the business simultaneously and provides an excellent opportunity to tap into the  knowledge of others. Also, working in cross functional teams can be very rewarding by providing a lot of variability to the job and making it more pleasurable.
[bar group=”content”]

Demand Planning Is A Collaborative Process That Provides Visibility, And Opens Doors

It is a collaborative process which aids in developing many relationships through the internal organization, as well as, customers and other suppliers. It is a highly visible position which can lead to new and exciting projects. Along with the knowledge to be gained from these groups, the relationships become an asset for your forecasting success and in turn your career path. Also, it can be very rewarding to work with other people to help them attain their goals and reach a collaborative decision that will benefit the entire company. A successful demand planner must become a leader in fact based decision making and a champion for change.

The Required Leadership In The Role Is a Challenge

Along with business knowledge and relationship building, leadership skills are also an asset to a successful demand planner. A successful demand planner uses the knowledge gained and is able to interact with customers, managers, sales representative, marketing, pricing and supply chain colleagues. Becoming a good communicator is imperative to collaboration among internal and external customers. This will enable the demand planner to guide various groups in terms that make sense to them and to reach consensus among the group. All of these things together help the demand planner to provide the best forecast possible which in turn will become a huge advantage for both the company and the demand planner.

Ultimately, a bad forecast leads to bad corporate decisions and the loss of career possibilities. Take the positive approach using business knowledge, building relationships and leading your colleagues to collaboration. Pave the way for fact based decisions that will benefit you and your company. Don’t become a victim and fall for the curse. I have learned over the years to approach my career and my life with gratitude and a “can I help you” attitude. This will take you farther than any expertise on any day. Curse or Blessing, well maybe it was best defined by the Beatles, “I get by with a little help from my friends.”

Sylvia Starnes
Demand Planning Leader
Continental Tire

It all started at AIWA Consumer Electronics in the early 1990s. Here I was, a fresh MBA in my first Associate Product Manager role for personal audio products. Every year, new versions would replace the prior year’s products. End of year was filled with clearance specials of products just sitting in our warehouse that had failed to sell. The more that accumulated in warehouses, the further away we were from our profit targets.

One of my product marketing responsibilities was the demand plan. The inherited budget and forecast was prior year sales + 10% expected growth rate on all designs. My counterparts covering other categories used the same approach, and worked at selling the current inventory. It was a basic approach, to put it kindly. This is a story of how that paradigm changed completely.

Overall forecast error at the time was around 20% for the 5-month order-to-warehouse cycle. Not bad by marketing team reasoning but nowhere near good enough by management’s standards, or ours come to think of it. Collaboration meetings would open with our distribution manager, Joe Sparta, saying “You guys are selling plenty of what we don’t have and not enough of we do have”.

The lack of insight into what our customers wanted and how to service that need was painfully obvious. The inefficiency in supply chain was shocking.

In graduate school, I had the pleasure of taking Chaman Jain’s graduate forecasting class where I learned the power of measuring MAPE. I applied this method to my new role and running the numbers by SKU at AIWA, we had 105% MAPE. No wonder we needed clearance specials, and Joe was having a difficult time with customer service!

Something needed changing, and fast.

Luckily, Chaman Jain was open to helping out an old student. He has an immense wealth of knowledge and experience, and I was fortunate enough to be taken under his wing. Chaman Jain founded IBF, is the author of the Fundamentals of Demand Planning book and is Editor of the IBF’s Journal of Business Forecasting. I couldn’t have hoped for a better mentor.

Chaman had been where I was at that time, and understood the challenges I faced. What’s more, he knew intimately all the mistakes we were making. What I was trying to do was desperately make sense of vast amounts of data that seemed more like a quagmire than a clear insight into customer demand.

We discussed the challenges of relating older products to the newer versions. Considering the seasonality of my category (Christmas, Dads & Grads bumps), he recommended starting with multiplicative decomposition. I took shipment seasonality of the category rather than prior products because of the changes in features at price point. The next step was treating distribution penetration as a trend level. The distribution multiplier became the retailer share of consumer sales published by the trade magazine TWICE (This Week in Consumer Electronics). This dropped my MAPE to the sixties. We weren’t where we wanted to be, but changes had been implemented and definite progress was made. Joe was happier, and we had put the company on track to greater profitability.

The next step to improve accuracy was to use NPD INTELECT POS data. Clark Johnson, the sales VP who provided our data, provided in-depth guidance on data limitations. Using consumer purchases, I fine-tuned these simple models as the data became available (there was a 2-month delay).

My first step in changing category seasonality from shipments to consumption dramatically improved the forecast accuracy after initial pipe fills. This meant while we still had a good call for the highest seasonal period of Christmas, we had a weaker call for Dads and Grads when POG (Plan-O-Grams) were being reset. Overall, our MAPE for the 5 month order period improved to under 40%. Joe was getting happier.

Sales and retailers were not as happy, however. The more efficient our supply chain, the fewer opportunities there are for discount clearances. My manager and VP were very happy that profits were up for my category. This resulted in a promotion to Product Manager of Personal Stereo, as well as assuming additional responsibilities forecasting the other 4 categories.

My move to a forecasting career was solidified by one product. Our team designed an innovative new product – the first 100 watt, simple to operate, shelf top stereo system. Before this, all US shelf top stereos were between 10 and 30 watts of power. The product and sales teams believed that focusing on lower watt stereos were holding back our US market share and that the new stereo would be a game changer.

Launching the new product was quite a gamble to say the least.

The initial forecast was to move AIWA from a 12% share to a 18% share of the shelf top category. The first hint of success was 100% distribution with all major retailers placing the product in their POGs (Plan-O-Grams). We sold out of the initial container loads as soon as they arrived and increased our forecasts.

The gamble had paid off big time.

Management’s key question was how big could this get? We did not know if retailers were loading inventory because of expected shortages (common at that time – see MAPE comments above). But when the consumption data came in, we knew we had a hot seller. Using two months of fully distributed consumption sales divided by the category seasonality of those two months, we had an estimate of between a 55 and 65% share of our projected category sales (compared to our budgeted goal of 18%). This was a huge success for the company.

We only doubled our forecast because management did not completely buy into the seasonality based forecast. When the third month of consumption confirmed the same 55 to 65% share, our senior management team had the Malaysia factory to go to triple shifts. We manufactured for the conservative 55% share number and ended up selling out of almost all the containers coming into the US, exceeding the quantity. AIWA grew the category and become the dominant category leader.

Forecasting’s role in driving the company from $100 million to $300 million sales changed my career. To say it was an exciting time is an understatement.

I saw the power of forecasting and I was hooked.

Soon after, Roger Brown recruited me to Duracell. He exposed me to the consumption paradigm with rich data sources to drive the forecast. As part of Roger’s team, we regularly attended IBF forecasting conferences. I now had the opportunity to work with, and learn from, some of the best in our profession.

It all started with a conversation with Chaman Jain and joining IBF. Forecasting through demand planning and applying predictive analytics is a vocation that still inspires, challenges and motivates me today. And, why wouldn’t it? When done properly, demand planning sales to retailers through understanding the consumer allows you to predict the future. It enables even established companies to reach unimaginable levels of profitability.

Scott Roy

[bar group=”content”]

Starting My Career In Demand Planning

A few years ago I found myself reviewing an Institute of Business Forecasting & Planning – IBF presentation that I was going to give the next day when I realized that I had used the word knowledge about five times in just the first few slides. I sat back for a moment and realized that I really didn’t know what knowledge meant? I had some vague understating; but what did it really mean?  I took a quick trip to Wikipedia for a definition; “The term knowledge is used to mean the confident understanding of a subject with the ability to use it for a specific purpose.” While I was there I looked up another word; “novice: a person who is new to a field or activity.”

Demand Planning Is Easy, Right?!

This caused me to even ponder more on how we go from novice to knowledge! I did not start my career planning to be in the field of demand planning. I actually started off in public accounting and then found my way to cost accounting then into information system and then into operations and supply chain management.  While I was working in supply chain, I went through the APICS certification process, where I read about forecasting and demand planning. When a job in demand planning opened up down the road, at a point in my life when I needed a new job, I figured demand planning; how hard could this be?  At my new job my book smarts said, put about three years of sales history into some forecasting software, sprinkle a little magic dust and out comes a forecast!

I created my first forecast that way.  However, being all smart and smug about what I thought I knew, I then proceeded to spend the next year figuring out how to fix all of the things I had done wrong.  I was a novice.  I wasn’t even smart enough to know what I didn’t know!  I sort of knew what to do, but had no clue why I was really doing it and what to watch out for. I think it’s kind of like golf, everyone thinks they can do it; you take a club and hit a ball into a hole…. how hard could that be?? This is the point in my career where I started looking for some real education on how to do demand planning & forecasting the right way and this is when I came across the Institute of Business Forecasting & Planning – IBF.

Be Wary of Fancy New Systems

Nine years ago from today, I started my demand planning journey from novice to knowledge. I started off on the cheap ordering some conference proceedings and then finally went to my first IBF conference.  I was learning from all of the mistakes that I had made, but needing to keep my job for a few more years; IBF really put me on the fast track to a better understanding of what I was really supposed to be doing! I found the combination of insights from consultants and real life experience from practitioners really made the difference. Going back to my golfing example, in golf you need to have different clubs for different situations, and know how to read the course.  However, for the most part, I’ve always tried to steer clear of consultants as they seem to be always trying to sell you a “new golf club” that you don’t understand how to use, which cost a lot of money!  But, to the contrary, I have actually become more tolerant with consultants, especially through my experience with Mike Gilliland a frequent IBF contributor and IBF Board of Adviser. He seems to care more about sharing his great knowledge of demand planning than selling you anything. (There! I went and used that word knowledge again as when it comes to Mike, he knows how to get the ball in the cup.  Plus, he knows how to apply his skills in a way to get those desired results!)

Today when it comes to my view of demand planning, I rely on using a few golf clubs very well.  I don’t do a lot of fancy things, just keeping it simple.  I do the things I can understand.  Another key thing about golf and demand planning, not all of the courses are made equal. Before you pick up a golf club, you must understand what you are up against; get a lay of the land and understand what the hazards are.  My first year on the job, I admit, I was  embarrassed.  Luckily, I was a quick learner with a lot of help from IBF and many dedicated professionals who were willing to share their knowledge of the “game!”

To fast track yourself from novice to knowledge, getting help from organizations like IBF really help for demand planning & forecasting.  It would be great to hear how you got into demand planning and forecasting?  It would also be great to hear your stories of going from novice to knowledge in your demand planning, forecasting, and supply chain experience….. the people and organizations that have helped you understand the game and take “stroke off your score!”

Scott Roy
Collaboration Planning
Wells Dairy Inc

Chaman L. Jain, Ph.D

[bar group=”content”]

No matter where you place the forecasting function, you will have a bias in forecasts unless you create an independent department. Production people tend to over-forecast because it gives them fewer headaches resulting from out of stocks. But if they are evaluated on the basis of inventory, they would prefer under-forecasts. Salespeople have a tendency to under-forecast where sales quotas are tied to forecasts. Otherwise, they would prefer over-forecasts to make sure products are available when orders arrive. For marketing people, it all depends. If the advertising budget is tied to forecasts, they would prefer over-forecasts. Finance people, in general, are conservative, but their mind-set may change when they report to Wall Street. Having an independent department is a solution but only large corporations can afford this option. So the question is not how to avoid the bias, but how to minimize it. A good consensus process with a good champion can help to reduce the bias. In that case it may not make that much difference where the forecasting function is placed.

IBF Benchmark – www.ibf.org

One way to make that determination is to see where different companies have their forecasting function. The figure below can give you an idea, which is based on the survey data conducted by the Institute of Business Forecasting and Planning – IBF. (Data are of all the industries combined). It shows that a large percentage of companies have their function in the supply chain (35% = 26% + 9%), followed by Sales (16%) and independent forecasting department (14%). I have been watching the survey data for the last eight years. Two things I have noticed. One, more and more companies are moving their forecasting function to the supply chain probably since this is where operational forecasts are used most. Two, more and more companies are moving their forecasting function away from Finance. The percentage of companies having their function in Finance has declined from 14% in 2001 to 7%. Where does the forecasting function reside at your company and why?  It would be great to hear from you!