It is a great time for business forecasting and planning and those who do it! We have technology and data analytics tools that could not have been imagined just 30 years ago. We have able to shift from manual, time-consuming data collection and analysis to enabling better business decisions quicker and easier, adding more value to our businesses.

It has been a time of great expansion of data, and of great advancement in forecasting tools to exploit it. We can better understand the purchasing behavior of customers and consumers. And we have better forecasting methods and models to generate demand forecast and revenue projections.

Demand Forecasters and Planners are now business partners in decision making across the entire enterprise – and that should be our goal.

Your Role Is Only Getting More Important

Operational and financial plans can are only as good as the underlying demand forecasts and plans. This places Demand Planners in uniquely important position, and one with great responsibility when it comes to the success of the company. Welcome to the 2020’s where Demand forecasting and demand planning are foundational. Welcome to a time of multi-dimensional business thinking. Having seen planning transition from a traditional approach of reverse engineering from the top line to the bottom-line performance, it’s like shifting from 2-dimensional chess to 3-dimensional chess.

Welcome to the 2020’s where Demand forecasting and demand planning are foundational

Shift Away From Short-Term Forecasting

What can we do to leverage the full range of capabilities available to us? Currently, a limiting factor for many is the continued use of spreadsheets. Spreadsheets are still widely used by Demand Planners in companies of all sizes. Spreadsheets require significant time to import data, maintain it, perform modeling, and export data for use in other software. Integrated software solutions within the company should be standard, freeing up time for Demand Planners to create insights of greater value to the company. Innumerable cost-benefit studies support this. Time is money.

It is important that time be dedicated to identifying and analyzing the drivers of demand

A Demand Planner’s time should shift from away purely performing short-term forecasting, for which Machine Learning (ML), and Artificial Intelligence (AI) can be readily deployed. If technology can assist in short term forecasting, what should we do with this extra time? It is important that time and effort be dedicated to identifying and analyzing the drivers of demand (and the forces that affecting these drivers), both now and in the future. This places the Demand Planner in a position to understand and anticipate bigger picture forces impacting the enterprise, and to inform and assist the strategic discussion and decision-making. That is enormously valuable.

Make Collaboration Your Superpower

Given all functions throughout the company plan and depend on demand forecasts and plans, development of relationships across functions is a major value-added. Become familiar with the terminology used by the functional areas participating in and using information from your demand forecasting and planning processes.

Help them to use the information you provide more effectively for their unique roles. Determine how you may be able to reshape information that you provide in a better format or segmentation structure. Ask them to share information with you from their functional area that may add to the effectiveness of the work in demand forecasting and planning, such as industry publications, research reports, market research, and other inside and outside information that they use.

Ask them about their views on events, situations, competitors, and other developments that impact their area of the business. Ask them how they might expect this to affect the company in the future. Ask them how it might affect their function in the future. Ask them to suggest conferences that they attend that they believe would add to the quality of the work that Demand Planners are doing. Ask and listen to inside and outside authoritative sources of information. Be genuinely interested in others and their functions.

Use Technology to Elevate Your Role to Business Partner

Implement technology solutions that enable better operational forecasts more quickly, while serving the company with insights and longer term forecasts that will drive strategic initiatives. Think of yourself as a trusted advisor for the company, aiming to deliver on enterprise goals and objectives and interested in its business success – and well qualified to guide the business in the right direction. This reframing of your role can enhance your value to the company and contribute to its future success and sustainability. Look constantly for ways to improve your functional area and the company as a whole.

Think of yourself as a trusted advisor for the company

As ML and AI develop in the coming years, they will likely be one of our best enablers and sources of rapid research and information for the work of demand forecasting and planning for future company success. The pace of change is rapid and unlikely to slow down in the future. Acquiring tools and applications that enable us to work more effectively and efficiently is essential. We should not fear technological developments but embrace them as they will be fundamental to our success, both as individuals and companies.

 There have been amazing changes for Demand Forecasters and Demand Planners in the past 25-30 years. And more change will come even faster in the next 5-10 years. It will be a great time and opportunity for business forecasting and business planning and for those who do it!

To get up to speed with the fundamentals of S&OP and IBP, join IBF for our 2- or 3-day Boot Camp in Miami, from Feb 6-8. You’ll receive training in best practices from leading experts, designed to make these processes a reality in your organization. Super Early Bird Pricing is open now. Details and registration.

It is nearly impossible to avoid the current hype around ChatGPT.  ChatGPT is an artificial intelligence content creator that generates a variety of outputs by answering questions fed to it.  You would have to be living under the proverbial rock not to hear the news stories – ripe with examples of the ChatGPT application passing MBA tests, and Law School exams.  The hype piqued my curiosity, so I decided to spend a cold February weekend testing out the application.  

Not knowing exactly how to use it – I decided to just have some fun. I prompted the application: “Create a short biography of Willie Mays”. It created two paragraphs of clean content highlighting May’s greatness as a baseball player.  I repeated the question and added “ In the style of Hemingway.” The response astonished me in its clarity and precision and likeness to Hemingway’s style. I asked other silliness such as “Visualize entropy” and it described an organized and disorganized living room as a metaphor for high and low entropy.

Many, many years ago on a planet far, far away…I wrote a paper for my high school American History class. The paper was meant to emulate a well-researched college thesis, with a minimum of 50 pages, proper citations, a table of contents, and other related components. We were instructed to choose a topic that had either a historical or futuristic focus. Having just read Alvin Toffler’s best-selling “Future Shock,” I chose to write about the pace of technological change and its effects on society.

The End of the White Collar Class?

“Future Shock,” published in 1970, outlines the political, social, and technological impacts of rapid technological advancement. Toffler predicted the decline of traditional industries and the rise of knowledge-based careers leading to a constantly evolving job market where successful workers must be able to adapt and retrain quickly to maintain their employability. He foresaw the trend toward remote work, the gig economy, the Internet of Things, and even the planned obsolescence of products. Toffler’s insights, published more than 50 years ago, have proven to be largely accurate.

As I played more and more with ChatGPT, I could not help but wonder if this new tool and others like it would have a dramatic effect on the knowledge workers of today—the people foretold of in “Future Shock.” I tried to contextualize the impact of the technology. Would this automation be a help or a replacement for many of these workers? Could it help with some known labor shortages such as those in the supply chain? What would happen to the coders, content creators, illustrators, web designers, writers, and countless others engaged in careers that will be affected by this new tool?

This was the eureka connection to my high school thesis paper, I recalled considering the possibility that some forms of planned obsolescence might include people – expanding on the notion of technological unemployment first articulated by John Maynard Keynes. After a long career in supply chain and manufacturing, I clearly understand how the advance of breakthrough technology has shifted work. I have watched automated picking machines replace workers in warehouses, and robots replace legions of factory workers. And we are now on the cusp of automated driving vehicles that might replace truck drivers.

Through all of this “progress” I never once considered that white collar workers, the knowledge workers, would be impacted by advancing technology. I thought they were “safe”. I always assumed that the workers most likely to be “technologically unemployed” would be the folks working on a typical manufacturing production line, where a machine could be built or programmed to replace their physical labor.

After experimenting with my whimsical prompts, I gave ChatGPT a series of supply chain prompts such as : “Explain S&OP in simple terms.” Here again, I was amazed by the app’s near-perfect and grammatically accurate answer. ChatGPT was not a digital toy hardwired for fun and it is not just incremental improvement. It is a game changer. I was so enamored with this experience that I posted to LinkedIn my story of querying ChatGPT about S&OP. A former colleague, a senior marketing executive with a FinTech firm sent this reply to me:

“Saw your ChatGPT post. I’ve already started using it to write byline articles, but I would not say that publicly– I’d get scorched by copywriters, editors, etc. I’ve learned how to feed it to get decent fodder up front, and then I clean it up and add more nuanced info. It probably cuts article development time in half.”

My former colleague confirmed some of my concerns. ChatGPT is such game-changing technology that even in its embryonic form has already replaced human workers while improving outcomes. I don’t for a moment think the myriad of content creators or editors will lose their jobs immediately, but I can’t imagine many are happy with this new tool. They may eventually have to re-learn and re-tool to remain employable.

ChatGPT & Supply Chain Management

Channeling Toffler, I considered what this might mean in my own profession. What were the potential use cases within supply chain? Envisioning the possibilities of some natural language, quantitative ChatGPT “cousin” in the supply chain field, I can think of a hundred different ways I would leverage such a tool. As planners, we always search for that extra piece of data to help us perform our jobs more efficiently. Imagine someday using an app to inquire, “Where is the shipment of chemical X at the moment?” or “What is the forecast error for product Y?” or “Has product Z started being sold to Walmart?”

Imagine colleagues five years from now prompting their cell phones to “Generate a forecast for the new blue widget product line, using the red widget product line as an analog” or “Provide me with the economic and sustainability impacts of closing a warehouse in Memphis.” Then imagine a tool that could perform even more complex analyses: “What is the risk profile of supplier A?” or “What are the economic tradeoffs of a less than 100% fill level while serving Amazon?” When you consider all the data analyses that supply chain professionals perform on a daily basis, the opportunities for supply chain AI tools are limitless.

There is still much to learn as natural language artificial intelligence tools expand into many domains. To me, this is the real promise of Moore’s law – expansive computing power that provides digestible information at the speed of thought. The future is unknown (despite Toffler’s knack for prescience), but I suspect we will be talking about this breakthrough moment and its impacts for a long time.

Have you ever heard the Chinese proverb about the farmer? I originally heard it on a podcast featuring Dan Bilzerian.

The story goes:

A farmer and his son had a horse who helped the family earn a living. One day the horse ran away and the farmer’s neighbors said, “Your horse has run away, what terrible luck!” “Maybe,” replied the farmer.

Sometime later, the horse returned with a group of wild mares. The neighbors then said, “Your horse has returned with several other horses, what great luck!” “Maybe,” replied the farmer.

Later that week, the farmer’s son was trying to break one of the mares and she threw him to the ground, breaking his leg. The neighbors said, “Your son has broken his leg, what terrible luck!” “Maybe,” replied the farmer.

A couple of weeks later, the army marched through town recruiting young men. They did not take his son as he was recovering from his injury. The neighbors said, “Your boy was spared, what great luck!”

“Maybe,” the farmer replied.

The point of the story is that you never really know if something is bad or good because you don’t know how it’s going to affect the next step in your life.

I think about this when it comes to the future of demand planning as it relates to Artificial Intelligence (AI) and Machine Learning. How is it going to affect the next step in our career or even our supply chain?

Will AI change demand planning for the better? “Maybe”

AI-Generated Insight Isn’t Enough

About a year ago Daniel Fitzpatrick wrote an article titled Beware the Pitfalls of AI in Demand Planning, published on this website.

He mentions a few concerns but the one that caught my eye was “Forecasts as Proxies for Success”. In the article he says “Forecast accuracy is only a proxy for improved business performance. Without an effective supply chain to support more accurate forecasting, much of the value that an advanced algorithm might add may be lost. An excessive focus on improving forecast accuracy may draw attention and resources away from other constraints that are actually causing larger problems.”

“An accurate forecast does not reduce demand volatility.”

Let’s think about this for a minute. So even with advanced algorithms and heavy investment in AI, an accurate forecast does not reduce demand volatility. To tackle that, you are going to need to understand the root cause, some of which can be prevented and some are completely out of our control. Let’s separate the two.

On one hand, you have the preventative. Communication is at the top of the list. It’s not going to help when your sales team keeps market or customer knowledge to themselves. How about promotions and if they aren’t planned out correctly? They could drive bad consumer behavior, a self-inflicted wound of huge swings in demand. A third is inventory levels. Whether it’s through safety stock or customer inventory-level agreements, with the right strategy, both can help lead to smoother, more forecastable demand.

“Machine Learning Algorithms can only take you so far.”

On other hand, you have the uncontrollable. There’s weather, labor market and wages, raw materials costs and availability, and industrial production specifically as it relates to commodities. The list goes on, a ripple effect that affects all of our supply chains. The majority of my career has been in the CPG space and as much as I would like to claim to say I have seen it all, I am sure you could come up with a story that tops mine. The point is, AI and Machine Learning Algorithms can only take you so far.

“AI improves forecasts based on real-time, internal and external data”

Demand Planning isn’t simple and we can all envision putting ourselves on the imaginary process line of evolution ranging from the basic to the advanced. But unless you are on the top of the food chain, AI and machine learning can likely help. Machine learning can take you to the next level; it enables improved forecasts based on real-time data using internal and external data sources and can turn the uncontrollable into the measurable.

External Data Is Our Friend

External data is our friend and modern machine learning algorithms combined with our supply chain networks can likely outperform processes that are managed solely by Demand Planners. Think about new products. What if AI helped users identify products with similar characteristics ultimately leading to better predictions? Demand Planners can be turned into Super Demand Planners.

Recently I have been getting the impression that there is going to be a shift in our world of Demand Planning. Maybe it’s about finding efficiencies in processes, maintaining lean personnel, or maybe it’s about preserving inventory levels and increasing cash flow. Either way, change is coming and likely in the name of AI.

So, is Artificial Intelligence (AI) to the Rescue for Demand Planning? Maybe.

Maybe we can expand our set of tools and work smarter, not harder. Maybe there is always going to be the uncontrollable and too much data isn’t always a good thing.

The integration of Artificial Intelligence and Machine Learning into the S&OP process allows companies to become more agile in response to changes in demand. Data sources that previously were hard to analyze due to complexity or sheer volume are becoming standard planning inputs.

For example, using these tools, demand planners can now use data from sources such as online reviews, competitors’ advertising, and even Tweets.

With all that AI/ML can do to enhance the planning process, does this mean that the current role of demand planning is doomed? Should those of us who are currently working in the demand planning arena begin looking for new jobs?

Not so fast. While AI and ML offer our planning processes new and powerful ways of managing inputs to demand, they also have some significant limitations. And I believe human Demand Planners will be required to ensure that AI and ML are truly effective in the planning process. To better understand how these technologies and human Demand Planners can complement each other, let’s begin by looking at some of the often-overlooked limitations of these tools.

1. Patterns, Patterns Everywhere

The primary way that these tools can assist us in planning is by finding patterns in large amounts of complex data. As companies gather more and more data about their customers and their businesses, the quantity of data that they need to analyze to make good decisions becomes more than human Demand Planners can manage. By training AI/ML systems to find patterns that are buried in these mountains of data, companies can exploit data that was previously inaccessible due to volume or complexity.

But patterns can be a trap. Just because a customer bought a large quantity of product every September for the last 6 years does not mean that this will happen again this year. We still need to assess other factors that might influence this pattern such as pricing, product features, and competitive products. So, while AI/ML are good at indicating that these sales might recur, it will take some human input and research to determine if it makes sense to bet on it happening again this year. In time we may be able to train these systems to incorporate these potential sources of input; but in the meantime, we will need human insights to fill in these gaps.

2. Intelligence Vs Common Sense

Where human beings can infer things from common sense, AI/ML can be stumped. Look at this scenario: A man went to a restaurant. He ordered a steak. He left a big tip. If I asked a friend what this man ate, he would say a steak. But most AI/ML systems would struggle to get this answer because nothing in these statements explicitly describes what the man ate, only what he ordered. From experience, we know that what we order in a restaurant is usually also what we eat. This sort of common-sense extrapolation based on context is difficult for AI/ML systems.

In a planning scenario this can be a major problem. For example, a customer always orders a large quantity of a certain product at Thanksgiving, and later returns about 20% of the product since it has not sold. Does this mean that the customer doesn’t understand how to purchase product correctly? Or does it mean that they need excess quantities to ensure that their displays are full throughout the selling season? While AI/ML can’t answer these questions, human planners can contact the customer and asses what the real issue might be.

3. AI Has Limited Adaptation

One of the strengths of human intelligence is that the human mind can easily adapt to new information. If I tell you that a customer just went bankrupt, from experience you will know what impact this might have on your business. You can quickly adjust your processes to accommodate this change. AI/ML can’t react that quickly. These systems would need to be retrained to know what to do in this situation.

And since each situation would be slightly different, any training provided for one scenario would have only limited application to later ones. These systems need ongoing training to be truly agile and adaptive.

4. AI Has No Understanding Of Cause & Effect

Humans from experience instinctively understand cause and effect. If I drop a glass on a hard floor it will shatter. But my dropping the glass does not cause it to shatter. The glass striking the floor causes this. Here’s another example: We know from experience that roosters crow when the sun rises. AI/ML have no trouble understanding this relationship. But if we ask whether the rooster’s crowing causes the sun to rise or vice versa, these systems are stumped.

Using the customer bankruptcy example above, from our experience we can usually easily assess the possible causes: lack of sales, high expenses, loss of funding, better competition, etc. Our experience allows us to make these mental jumps easily. However, without extensive training, AI/ML systems would struggle to relate the causes to the effects.

5. AI Lacks Ethics

AI/ML systems will reflect the biases and perspectives of the humans that trained them. They can’t tell right from wrong. Programming these systems to reflect the complexity of human values and how these adapt to different and changing situations is extremely difficult. Therefore, allowing them to make certain types of decisions can be dangerous.

For example, in planning how much credit to extend to a customer, we can train a system to analyze the business factors that make a customer a good or a bad business risk. But these systems can’t tell us how well these customers manage their environmental or societal impact. If these factors are important to our decision, we need human input.

Complementary, Not Competitive

With the limitations of AI/ML that we have discussed here, it might seem that these tools are less useful that we might at first have thought. The truth is that they are extremely useful when we are dealing with large amounts of data, and where we have the time, skill, and resources to train them properly. What they lack is what human planners can provide. In the best case I believe the combination of properly trained AI/ML systems and experienced Demand Planners can be extremely effective in drawing out all the insights hidden in the data.

To make the most of this relationship, demand planners will need to develop some new skills. While we can leave a large part of the data analysis to systems, human insights based on broad experience will be required if we are to make the most of the analysis these systems provide. Additional human soft skills such as relationship-building, listening, innovating, and thinking strategically — together with the input of AI — can make our planning both more agile and more effective.

This article originally appeared in the Fall 2021 issue of The Journal of Business Forecasting. Become an IBF member and get the Journal delivered to your door quarterly, plus discounted entry to IBF conferences and events, members only tutorials and workshops, access to the entire IBF knowledge library and more. Get your membership. 

I am going to attempt to pull back the curtain and unveil the magic behind the most common algorithms used in predictive analytics for business forecasting, and demonstrate what exactly goes on behind the scenes in the world of AI.

Here I focus on the top methods and algorithms that enable the execution of applications for demand planning and business forecasting. The following are the preferred Machine Learning and Predictive Analytics models of Demand Planners and Data Scientists (in reverse order):

7) Artificial Neural Networks

6) Decision Trees

5) Logistic Regression

4) Naïve Bayes

3) Linear Regression

2) Smoothing and Averaging Time Series Models

1) Simple Ratio Models

7) Artificial Neural Networks: (ANN) are a class of pattern matching techniques inspired by the structure of biological neural networks. ANNs combine logistic regressions into a neural network. ANNs less complicated than they may first appear  – ANNs are a collection of logistic regressions, so if you understand logistic regression, you can easily understand ANNs. Developing the final prediction is generally done by training the model and calibrating all of the “weights” for each neuron and repeating two key steps: forward propagation and back propagation.

6) Decision Trees: The concept and algorithms underpinning decision trees are relatively simple compared to other models. The general purpose of decision trees is to create a training model that can be used to predict the class or value of target variables. Decision trees build classification or regression models in the form of an upside-down tree structure.

5) Logistic Regression: These help demystify neural networks and help answer probability type questions. While it is listed as a type of regression model, it is less a calculation and more an iterative process. This can be used for anything from predicting failures to identifying if the object in a picture is a cat or not, for example.

4) Naïve Bayes: Naïve Bayes are probabilistic, which means that they calculate the probability of each class for a given set of features, and then output the class with the highest observed probability in the data set. In simple terms, based on a bunch of x’s, we’re looking at the odds of Y being y. It can be used for natural language processing, to classification, to simple prediction.

3) Linear Regression: Regression is a simple cause-and-effect modeling that investigates the relationship between dependent (target) and independent variables (predictor). Regression models come in all types and applications. One of the most common is a simple linear regression. The first step in finding a linear regression equation is to determine if there is a relationship between the two variables. This is often still a judgment call for many Demand Planners.

2) Smoothing & Averaging Time Series Models: With these models, one needs only the data of a series to be forecasted. They are simple and widely used by companies that own historical data. Here we assume that the training data set of sales history contains a trend and seasonality, and we extrapolate these patterns forward.

1) Simple Ratio Models: These express the relationship between two or more quantities.  Ratio models are utilized for a range of day-to-day purposes: understanding a seasonality index, calculating the velocity of sales and market penetration, or disaggregating a family level forecast, just to scratch the surface.  This easy-to-calculate statistic is used in various ways to guide decision making and drive forecasts.

What we find is that sometimes the simplest methods provide us a good forecast and are the best use of our time. Fancy techniques are great, but our overriding goal is to select the model that fits our business purposes and the resources available to us. We need to evaluate the model properly to ensure that it can do what we need it to. The most sophisticated techniques and most advanced technologies accomplish little if nobody understands the results. To complete the process we must step back, sometimes simplify, and communicate our analysis effectively. Or just tell them it was AI and leave them in awe  of your magical skills…

Don’t forget to join myself and a host of predictive analytics, demand planning, and forecasting leaders at IBF’s Virtual Business Predictive Analytics, Forecasting & Planning Conference from April 20-22, 2021. At just $499 for this insight-packed 2-day event, it’s an extremely cost-effective way to evolve your skills for the future of demand planning and forecasting.

To add the above-mentioned models to your bag of tricks, get your hands on Eric Wilson’s new book Predictive Analytics For Business Forecasting. It is a must-have for the demand planner, forecaster or data scientist looking to employ advanced analytics for improved forecast accuracy and business insight. Get your copy.

As we integrate artificial intelligence (AI) and machine learning (ML) into our demand planning processes, I am sure that we will see improvement in our ability to anticipate demand rather than react to it. However, accurate forecasting is only one element in an effective supply chain. So, while I am very much in favor of making good use of these new tools, I have some reservations about their impact on improved demand planning.

My concerns fall into five categories:

• The nature of algorithms
• Algorithms don’t execute
• Gaming and overrides trump algorithms
• Forecasts as proxies for success
• Implementing advanced algorithms may initially make things worse
• Implementing advanced algorithms may reveal significant supply chain inefficiencies

1. The Nature of Algorithms

Algorithms are models of reality, and all models fall short of representing reality with perfect accuracy. Understanding a model’s limitations is key to its proper use. Certainly, AI and ML algorithms will allow us to better model potential demand, but we will also need to be aware of their limitations.

“all models fall short of representing reality with perfect accuracy”

Bad data, incorrect or biased interpretations of the data, and ignoring data that does not agree with corporate direction remain significant risks in any planning process. Adding reliable processes to validate both the data itself as well as any interpretations of the data will improve the effectiveness of these advanced algorithms.

2. Algorithms Don’t Execute

Poor supply chain execution will undermine any algorithm, no matter how accurate it is. Relying on improved algorithms alone will not improve a supply chain that is riddled with ineffective practices and siloed teams. In fact, more accurate modeling may reveal just how detrimental these poor practices are. An accurate forecast that correctly anticipates future demand will be worthless if the product can’t be produced and shipped in time to meet the demand.

And a more detailed view of future customer behavior will be worthless if the company cannot focus the necessary resources on planning the development, production and shipment of products that satisfy the customers’ expectations. Adding performance metrics to key supply chain processes will allow for discovery of potential constraints. And setting up processes to address these constraints as they appear will allow for continuous improvement throughout the supply chain.

3. Gaming and Overrides Trump Algorithms

A reliable algorithm that no one trusts will be of little value to a company. When individuals or teams believe that their view of the future is more accurate than a system’s predictions, and they are allowed to game or override the algorithm, most if not all of the value of the algorithm is lost. In my experience, most companies have a significant number of S&OP team members who distrust the systems they use to plan.

“A reliable algorithm that no one trusts will be of little value to a company”

Unless this is addressed, this underlying lack of confidence in any system will severely limit any algorithm’s impact on improving forecast performance. Overrides should be documented and validated by product performance and gaming should be clearly discouraged and called out when it does occur.

4. Forecasts as Proxies for Success

In applying AI and ML algorithms to our business, we need to ask what our true goal is. Is it really a more accurate forecast? Or is it a more robust and agile process for responding to customer demand?  It is possible to improve forecast accuracy without also improving service levels and on-time delivery.

Forecast accuracy is only a proxy for improved business performance. Without an effective supply chain to support more accurate forecasting, much of the value that an advanced algorithm might add may be lost. An excessive focus on improving forecast accuracy may draw attention and resources away from other constraints that are actually causing larger problems.

5. Implementing Advanced Algorithms May Reveal Significant Supply Chain Inefficiencies

There is no guarantee that implementing advanced forecasting algorithms will improve business performance. A more accurate forecast may reveal that the company can’t actually respond quickly as market and customer preferences change.

It may also show that more resources will be needed to support the execution of a more accurate forecasting process. In the long run these are useful lessons that the company can use to address constraints to improve the entire supply chain to take advantage of more accurate forecasting. So the expectation needs to be that these advanced models will be part of a continuous improvement process that will require all the links of the supply chain to become more effective by addressing constraints as they are discovered.

6. Stepping Back to Step Forward

The success of any supply chain is dependent, in large part, on the people in it understanding their roles and executing effectively. As AI and ML models are more integrated into the demand planning process, the key practices of good communication, ongoing training, executive support and continuous improvement will also need to be supported. Without these basic practices, the value of improved forecast accuracy may be limited.

By themselves these new algorithms will only show us what is possible. It will be up to each member of the S&OP teams to make sure that the possible consistently becomes reality through consistent execution guided by reliable performance metrics.

To find about more about practical applications of machine learning models, pick up a copy of Eric Wilson’s new book, Predictive Analytics For Business Forecasting & Planning. Written in easy-to-understand language, it breaks down how machine learning and predictive analytics can be applied in your organization to improve forecast accuracy and gain unprecedented insight. Get your copy. 

The world of forecasting today is filled with machine-related buzzwords like AI, predictive analytics and machine learning. Will all of forecasting be done by machines a few years from now? Do humans still have any competitive advantage over software? In the realm of new product forecasting – the human wins.

Human trumps machine forecasting in the following areas:

• Measuring promo impact
• Gauging life cycle trends
• Disentangling overlapping marketing tactics
• Incorporating qualitative analysis

Machines have great potential to improve forecasting processes. Automating forecasts for existing products that have a lot of data can free human time to focus on new product forecasting. However, engaging software in forecasting all products, both old and new, can pose serious risks for forecast accuracy.

How Much Value Does AI Really Add?

Hollywood has been questioning whether the machines will take over the world for almost half a century so it must be true. How will it start? Machines flipping burgers, executing court rulings or maybe doing demand planning? Demand planners are naturally intrigued about the latter. Industry trend-setters prefer to talk about AI fueled by machine learning algorithms over Excel books fueled by ridiculous amounts of coffee. Buzzwords sell, but do they really add value?

To answer that question we need to ask what AI and machine learning are. In the context of forecasting, these disciplines are essentially a series of algorithms that create baseline models and measure promotional impacts. The ‘machine learning’ component is a fancy term for the trivial process of feeding the algorithm with more data. These tools are very useful for forecasting products with lots of history and homogeneous promotions. These types of products are usually the easiest to forecast. Forecast error for products with lots of history is typically low, regardless of the level of automation. The tools save a lot of time and brain cells from doing tedious repetitive tasks, but they hardly improve forecast accuracy.

Can Algorithms Work With Little To No Sales Data?

But what about new products that have no or little sales data? Ironically these are also the ones that are subject to all kinds of marketing extremities. Can any machine algorithm forecast those? The short answer is no. And this is why:

1. Measuring promo impact as % vs Baseline may be misleading

How does software measure promo impact? Most of the time it would just compare actual sales to the baseline. % difference would be the promo impact that the machine will use to forecast future promotions. However, reality might be a bit more complex than that. Imagine you are forecasting a winter ice cream promotion based on a previous summer promotion. Your summer baseline was 100 units and you sold 200 units as the result of the promotion. The software calculates a 100% lift. Impressive right? Let’s apply that to your winter baseline of 15 units. Not that impressive anymore. A human can create a promo lift that might not be calculated by a particular formula but nevertheless is reasonable.

2. Product life cycle trends can throw machines off

Most products have a sustain period after the launch. This is the period of the product life cycle right after the initial launch when sales gradually decline until they level off and start exhibiting seasonality. Software is not very efficient at forecasting this trend. It usually either underestimates future sales by applying the negative trend indefinitely or overestimates by applying seasonality to sales that are still elevated after the launch. A human can apply common sense to determine when the sustain period ends and normal seasonality starts.

3. Disentangling components of promo impact via algorithms may result in lower accuracy

Every new product promotion is unique with its own mix of marketing support, pricing and product strategy. Imagine you are forecasting a promotion based on two historical promotions. One historical promotion had a 10% lift and the other a 100% lift. An algorithm would use the average 60% lift for the forecast. In most cases this would be inaccurate. A human can make a judgement call on what particular factor differentiated the two historical promotions and choose the most similar one as a proxy for the forecast. I know what you’re going to say, “But we can teach the machine to recognize different kinds of promotions!” This brings me to my next point…

4. Teaching machines may be more work than doing it yourself

When a human analyses historical promotions he/she can look at different files, process different formats and even ask for other human opinions. A machine, however, demands to be spoon-fed data in a consistent format from one source. Creating a variable for every marketing strategy for every historical promotion can be very time-consuming.

5. Are you working on a forecast or on a model?

A statistician will say that all of the above issues may be resolved within a model by overriding product life cycle trend, flagging different kinds of promotions, using proxy seasonality, etc. This is true. The caveat is that these processes require a lot of human hours. Human hours dedicated to perfecting the machine, rather than perfecting the forecast. Anyone who has worked with models knows that statistical significance does not always go hand in hand with common sense. Overrides and interventions increase the risk of over-specification of the model. This means that model results stop making sense because there is not enough data to support all the additional variables.

6. Humans can think outside the box while machines are the box

Forecasting new products that have no data is solely reliable on proxy selection. Imagine you are forecasting a new LTO flavor of ice cream – dark chocolate. You have history on two LTOs – coffee and caramel chocolate ice cream. Which flavor will you choose as the proxy? A human will consider a plethora of factors. Which of the 2 historical flavors was more similar in taste? Is the bitterness or chocolatey-ness driving the sales? Is there any consumer research on the two flavors? Which of the promotions had similar marketing support? Was the creative more appealing for one of the historical flavours? All these questions belong to the qualitative, not quantitative realm. Hence, the software will not be helpful.

So what is the right balance between human and machine in forecasting? The answer is obvious – leave the quantitative tasks to the machines and qualitative to humans. The problem is that there is a fine line between the two in the realm of forecasting. The best term for a forecast in my mind is ‘a scientific guess’. We don’t want to leave the guessing part to the machines as it relies on intuition – an inherently human feature. It is worthwhile investing time in the exercise of classifying forecasts into ones that are generic enough to be done by machines and the ones that require a human touch. Machine and human can even efficiently cooperate within one forecast. For line extensions, software can predict the baseline level and even the promotional impact for the total line. Human can decide what proxy to use for the new flavor.

New product Forecasting Software Not Worth The Price

The good news is that software can effectively and efficiently forecast products with lots of history and homogeneous promotions. And they’re not expensive and are very easy to operate. There are even add-ons in Excel that can create baselines for hundreds of SKUs at different levels – provided you have enough data. It is the software that claims to forecast new products that can get pricey while providing dubious value add.

Demand planners needn’t fear being replaced by machines – only humans can provide valuable qualitative judgement.

Reconciling Quantitative & Qualitative Is Key In new Product Forecasting

Automation can release human time that can be spent on increasing forecast accuracy on the products that are the most difficult to forecast. There is a lot of room for improvement in the realm of forecasting. Average error for new food is around 50%. This equates to lost profits and excess inventory. One third of global food production is wasted, a problem that software alone will not resolve. Perfecting qualitative methods of forecasting is as important as quantitative, and even more important, is developing methods for reconciling the two. There are numerous consumer research specialists in the retail industry selling their qualitative expertise on new product appeal. Developing a system that converts such consumer research into real numbers that can be used for forecasting can significantly improve forecast error.

Artificial Intelligence has been widely praised for its ability to transform the supply chain for the better, from more accurate demand forecasting to real-time, automated inventory management. Zeroing in on that critical component of demand planning, an accurate forecast, AI-powered forecasts outperform traditional models, in some cases driving double-digit percentage gains in accuracy.

Sounds like a silver bullet, right? And it is. So why are many businesses still operating with poor, outdated forecasting processes? In IBF’s June 2018 survey, 70% of respondents said AI will be the future of demand planning in 2025. Forecasting practitioners are well-aware of the benefits, but the pitfalls of AI forecasting projects are preventing widespread adoption. Without coding or data science skills, it can be difficult for planners, analysts, and others tasked with improving forecasting processes to implement the most advanced AI-powered forecasting solutions.

Do Demand Planners Need Data Scientists To Exploit The Power OF AI?

When a demand planner has a centralized data science organization to collaborate with they have an opportunity to access the skills to exploit AI but typically, development cycles are long, costly and complicated. Many projects fail completely, often with a hefty price tag. At a leading Fortune 500 company I work with, the planners on the forecasting team are currently debating whether to rely on the company’s centralized data science team or hire their own data scientists. The strategic discussions have focused on these two options. Now, with new software platforms that automate the process of building AI models, they have a third option of incorporating advanced AI—Automated Machine Learning (AutoML) and Automated Deep Learning (AutoDL)—into their workflow.

These allow a demand planning function to achieve the same objective that they would get in options one and two, but faster and with less cost and risk associated with the project. The forecasting practitioners and demand planners don’t necessarily have a data science background but that’s not a limitation as these platforms empower them to take advantage of the latest in AutoML/DL to create process improvements and drive forecast accuracy—not in 2025, but now.

How Deep Learning Solves Common Forecasting Challenges

Tools powered by AutoML and AutoDL not only solve common, complex demand forecasting challenges with ease, but do so in a fraction of the time. Manufacturers of fresh, perishable goods create their own algorithms that precisely forecast daily sales for each SKU in each store, reducing wasted inventory and creating efficiencies in transportation and logistics. Apparel brand manufacturers are utilizing these platforms to improve forecast accuracy for new product introductions that have little historical data, mitigating risk in a way not possible with traditional forecasting methods. Fast food chains are predicting sales of individual menu items down to half-hour increments with greater accuracy than ever before, in turn allowing for better staffing and inventory management.

The biggest factor impacting accuracy is the data, and unlike traditional methods, Machine Learning and Deep Learning-powered methods allow forecasters to incorporate different types of data, structured or unstructured, and combine those data sets into the forecasting model. This holistic approach incorporates all data that could impact future sales, leading to major gains in accuracy.

Automation Reduces Tedious Manual Work

Automation is a critical component with any Machine Learning or Deep Learning software as without it, development processes are often still lengthy and failure rates high. AutoML and AutoDL forecasting platforms can automatically factor in seasonality, holiday and trend effects removing the need for tedious manual work for planners.

AI Helps Forecast New Products

While Machine Learning has provided many improvements over legacy demand forecasting methods, Deep Learning is uniquely well-suited for forecasting. Deep Learning acts like a long-term memory function and is better at learning patterns over time, and when historical data is incomplete or non-existent, like forecasting new products, AutoDL excels by quickly iterating to fill in these gaps.

AI Does Something Statistical Models Can’t Do – Identify Non-Linear Relationships

Deep learning solutions have greater capabilities to handle unstructured data and learn the correlations by understanding complex non-linear relationships. Sophisticated models are better suited for this type of task, however these advanced Deep Learning and AutoDL models require significant processing power. Largely made possible by GPU (Graphics Processing Unit) computing, these advanced forecasting algorithms can discover much finer patterns in the data than traditional methods.

AutoML and AutoDL solutions put the power of these unique capabilities into the hands of non-technical users while also solving pain points for data science personnel, like automating feature engineering, and model design, training and deployment.

When considering how to best incorporate the latest AI technology into forecasting, practitioners are no longer restricted to either partnering with an internal data science team, hiring their own, or simply relying on traditional methods. AI platforms that leverage AutoML and AutoDL can be valuable not only for data scientists but also for those business users who are fluent in data but perhaps not coding. This new breed of software is putting the power of Machine Learning and Deep Learning directly into the hands of planners and analysts, so they can build better forecasts today without the overhead, creating more accuracy and better decision-making throughout the supply chain.

IBF’s 2018 research report entitled The Impact Of People and Process On Forecast Error revealed that forecasting error of cause-and-effect models (AI) is more than four percentage points lower than Time Series models (20.79% vs. 24.86%). Click the link to download. 

While the basic concepts of Machine Learning (ML) and Artificial Intelligence (AI) are not new to forecasting and demand planning, there obviously seems to be renewed interest. For years, forecasters have used algorithms including artificial neural networks, association rules, decision trees, and Bayesian networks – all of which are common methods in Machine Learning. I guess you can say we were data scientists before data science was sexy. 

But this is not your daddy’s AI, in part because definitions change over time. Artificial Intelligence is the concept of machines being able to carry out tasks in a way that we would consider “smart”. There was a time when a machine could add 2+2 without humans providing the answer, and that was considered to be smart.  Now it deciphers unstructured text analytics in almost real time, tells us directions, makes predictions on our purchase preferences, and even talks to us.

Artificial Intelligence In Demand Planning Has Hit A Tipping Point

Another reason we are seeing this hype about Machine Learning and AI is we have also hit a tipping point. For a while, forecasting and demand planning processes and capabilities were greater than the technology that could support it. Now, we are no longer playing catch up and technology has surpassed planners’ abilities. Because of new technologies, the Machine Learning we see today is not similar to the type of Machine Learning we saw in the past. While many Machine Learning algorithms have been around for a long time, the ability to automatically apply complex mathematical calculations to Big Data – over and over, and at faster speeds – is fairly recent, and is now far, far more advanced.

There is an arms race to leverage both Machine learning and Artificial Intelligence in demand planning solutions

This has lead to an ongoing arms race to leverage both Machine learning and Artificial Intelligence in demand planning solutions more effectively, and in new ways. Interviewing one software provider, they stated that almost all their research dollars going forward are tied to these technologies.  All of this makes one pause and think that if we have come this far today, where will we be tomorrow?

IBF’s Survey Results Are In – Here’s What You Said

Last year the Institute of Business Forecasting (IBF) conducted our own research and asked demand planning and forecasting professionals how they saw their roles tomorrow. These questions were to measure how practitioners saw the discipline of demand planning in the year 2025 in regard to people, process, and technology. [Ed: this is the final in a series of articles discussing the results – other survey insights can be found regarding PEOPLE and PROCESS]

To help gauge the future of technology, we asked “What are the top technology advancements in the next 7 years that will have the largest impact on forecasting and demand planning?” Not surprisingly, of the 200-plus respondents, close to 70% of them answered Artificial Intelligence and Machine Learning as either their first, second, or third choice. Other key takeaways were dynamic simulation with 14% of first choices and blockchain (just making the list with only 2% of first choices).We are seeing these and other similar trends begin to emerge today.

70% of respondents considered AI and Machine Learning to be the most important technological advancement in 2025.

The Real Applications Of AI and Machine Learning Today

Research just released, conducted by Forrester Consulting and commissioned by Ivalua, revealed that 55% of organizations are set to make a major investment in Artificial Intelligence over the next two years. With applications using Artificial Intelligence and Machine Learning, algorithms can learn by taking the output of an application like a forecast and examining that output against some measure of the truth, and then adjusting the parameters involved in generating the output forecast, and seeing if the adjustments lead to more accurate outputs. Technology should also help liberate current demand planners away from rote SKU monitoring, toward more complex tasks, such as algorithm generation for the automated technology.

Starbucks will now generate serving recommendations for customers approaching their stores

Companies like Merck can change forecasts immediately based on far-off events, like hospital fires or natural disasters, that could impact demand for pharmaceuticals. Starbucks will now generate serving recommendations for customers approaching their stores, using location data to know when customers approach. One large CPG company is now considering lights-out statistical forecasting and re-deploying its demand planers to new value-added roles being more collaborative and supporting multiple functions and predictive analytics needs.

The research findings are taken from Institute of Business Forecasting’s (IBF) on-line survey “Future of Demand Planning and Forecasting”, conducted between September 1st 2017 and October 24th 2017. The findings were discussed in the Winter 2017 issue of The Journal of Business Forecasting. Click the link to download. 

If you would like to contribute an article to Demand-Planning.com, submit your details and suggested topics to the editorial team here.

Predictions are big business. The market for predictive analytics is expected to reach $12.4 Billion by 2022 as Artificial Intelligence (AI) becomes more widespread in Demand Planning organizations. But we’re not losing our jobs – don’t worry – because even though a lot of the analytical work will be automated, new roles are emerging to implement and manage AI, and interpret the findings. Here are the four types of AI in Demand Planning, and the new roles required to leverage them.

1. Limited Memory AI

Today, most Limited Memory Artificial Intelligence (AI) applications rely on an age-old premise: use vast stores of historical data together with current observations to predict future outcomes. A Demand Planner calculating the base forecast on a product is likely to use past sales to trend as a starting point. Depending on the number of products in a portfolio, this task could be executed efficiently without the use of AI applications, say for instance in a small enterprise. However, for large organisations such as multinationals, Limited Memory AI could execute this task with speed and accuracy providing live or on demand recommendations for review by the Demand Planner.

For any organization, investments in technology must provide a Return on Investment, with any decisions on AI implementation first requiring an extensive review of data stores to ensure they meet the required standard. Garbage in garbage out still applies and if you want quality data, you need a human.

Working In Reactive AI: Demand Planners with an interest in the technical aspects of the role may specialize as Demand Planning Scientists with a focus on Data Management and algorithm development, optimisation and maintenance. This function would support the initial preparation of data, selection and fine-tuning of the organization’s specific algorithms.

2. Reactive AI

Reactive AI, a second type of application, could also support Demand Planners. Scenario Planning and one-off decisions, which combines multiple information sources together, benefits launch and promotional planning where historical trending is not possible. Reliability of this proposition would depend on the extent of connectivity between the organization and it’s customers or key partners. Inputs could be drawn virtually from various stakeholders when scenarios are triggered as part of automated workflow processes.

Working In Reactive AI: Demand Planners could specialize in this area as Launch & Promo Planners. Successful collaboration with other departments has always been a differentiator of top talent in this area. Weak AI deployed in Limited Memory and Reactive applications do not offer direct substitutes for this aspect of human capability.

3. Theory Of Mind AI

Developed social characteristics of human communication and trust have proven difficult to replicate in AI. The Theory of Mind category of these applications deliver outcomes based on predictions of how people will behave within a given environment through perception of that environment and those within it. Where technological advancement of this type enables AI to read body language and interpret non-verbal cues to inform decision-making, opportunity may arise in collaborative forecasting processes.

Working With Theory Of Mind AI: AI assistants trained to work with teams may be well placed to provide a second opinion. Managerial and strategic skills would enable Demand Planners to thrive in such scenarios. Those individuals with the required capabilities may be qualified to be promoted to Demand Manager, overseeing AI decision recommendations, validating them and making the last call.

4. Self-Aware AI

AI in the fourth category is differentiated from the others based on its self-awareness capability, which draws on internal concepts of feelings and uses these additionally in the outcomes produced. This type of AI does not exist in the full capacity of that category definition, although strides are being made. To replace Demand Planners, AI would need those skills and more. Instinct, intuition, creativity to generate new ideas, social energy and charisma to influence are characteristics humans themselves are unable to quantify – far less replicate. These allow new ways of doing things and decisions to be taken outside of information that is available or could be generated. These include, for example, unprecedented events and rapid changes in consumer tastes including those for food or emerging fashion trends.

[bar id=”527″]

Working With Self-Aware AI: There’s is no new Demand Planner role for the type of AI (yet, anyway). That’s because in these circumstances, data does not usually exist. In professions where errors translate into costs such as lost sales and inventory builds, AI would excel in driving quality of information, automating mundane tasks, providing secondary opinions and recommendations. It could succeed under supervision, which would create a new Demand Planning role. How much time is committed to this would determine the viability of the application and the role. For those aspects of human nature, which enable social interaction, AI would fail and create a void in organizational cultures that could be detrimental to creativity, engagement and performance.

[For jobs in Demand Planning, Predictive Analytics and S&OP, check out IBF’s jobs board]