“What can be measured, can be managed.” This statement is certainly accurate with respect to the real value and purpose of using KPIs, or key performance indicators, as essential tools for measuring, monitoring, and managing process performance.

KPIs serve as benchmarks for identifying opportunities for optimization and innovation. They are of great use in decision-making, and are good instruments for creating accountability by setting clear expectations for execution. These indicators are mostly used to measure and evaluate performance against specific objectives or goals. For example, it is common for companies to establish KPIs as quantitative measurements of performance—to assess how well the organization is meeting its yearly objectives.

Even so, it is important to note that KPIs are not only limited to quantitative applications, as they can also be used to communicate good stories. Diligently analyzing and interpreting these indicators, far beyond comparing them to a target, enables the discovery of what they are truly communicating. By learning how to interpret KPIs, past events can be better understood and used as valuable evidence to predict trends, propose impactful business actions, and effectively communicate them across the organization.

Additionally, integrating and effectively managing KPI indicators through financial statements results, provides organizations greater visibility and improved decision-making processes that support their financial health, and enhance their ability to sustain long-term growth and profitability. For example, integrating Sales and Operations Planning (S&OP) KPI metrics, such as forecast accuracy, inventory turns, customer service level, supply chain costs, and working capital ratios, among others, to the Income Statement and Balance Sheet, allows businesses to achieve greater operational efficiency and financial results, which by effect, lead to sustained competitiveness and increased market value.

LINKING KPIS TO THE INCOME STATEMENT

The income statement is an essential financial statement that provides insights into a company’s economic position, profitability, and efficiency in generating revenues and managing expenses.

Income statement figures can reflect actions taken by demand and supply planning to make a positive impact on revenue, generate cost savings across various areas of the business, increase profitability, and optimize costs—such as selling and marketing costs.

Linking S&OP KPIs to the income statement, facilitates a direct understanding of how operational performance impacts financial results, and establishes clear correlations between them, thus leading corporations to strengthen their ability to make informed decisions that drive profitability and sustainable development.

From a revenue perspective, accurate forecasting ensures that the right products are available to meet customer demand, hence preventing lost sales opportunities. Higher customer service levels also drive repeated business, further boosting revenue. As such, improvements in forecast accuracy and customer service levels contribute to positively impacting the top line.

From the Cost of Goods Sold (COGS) viewpoint, effective inventory management reduces carrying costs associated with excess inventory, and minimizes the risk of obsolescence—resulting in lower COGS. Moreover, effective supply chain management facilitates negotiating better prices with suppliers, reducing procurement costs, and optimizing resource utilization. Overall, these cost-saving strategies directly impact the bottom line of the income statement by reducing operating expenses and consequently improving profitability.

The combined effect of higher revenue and reduced costs leads to improved gross margins. Gross margin improvement is a key indicator of the company’s operational efficiency and profitability, benefiting the income statement by impacting the company’s bottom line, while enhancing profitability and shareholder value.

LINKAGE TO THE BALANCE SHEET

The balance sheet is another essential financial statement used by organizations to provide a clear picture of the company’s financial position at a specific point in time. It presents the company’s assets, liabilities, and shareholders’ equity. While balance sheet items are not typically considered KPIs themselves, certain financial ratios and metrics derived from the balance sheet can serve as metrics to assess the company’s financial health and performance.

A typical indicator measured on the balance sheet is the efficiency of working capital management. Efficient planning techniques directly impact working capital management by optimizing the weight between current assets and liabilities. For example, improving forecast accuracy and inventory management reduces the need for excess working capital tied up in inventory. This by result liberates cash that can be used for other operational needs or investments, and at the same time, improves liquidity and financial stability, ensuring that the organization can meet its short-term obligations and/or invest in other significant growth opportunities. Another efficient planning technique is tighter accounts receivable management resulting from improved customer service levels that by effect can reduce the Days Sales Outstanding (DSO), further improving working capital efficiency.

Other S&OP KPIs such as inventory turns and inventory levels can directly influence the balance sheet as well. Higher inventory turnover ratios imply efficient inventory management practices, that lead to lower inventory levels, reduced excess inventory, and minimized carrying costs. Further, optimizing inventory levels reduces the risk of inventory write-downs and obsolescence, which can impact the company’s financial health and asset valuation. These reductions also free up cash for additional investment, and could even lead to debt reduction. Improved financial performance resulting from effective planning practices can positively impact debt management, enhancing profitability and liquidity for better debt management, reducing interest expenses and financial risk.

In summary, a strong balance sheet with healthy ratios, such as higher profitability and better liquidity ratios resulting from improved working capital management, can enhance the company’s creditworthiness, access to capital, and reduce its reliance on debt financing.

REPORTING KPI RESULTS TO EXECUTIVE LEVEL TEAM

Executives are mostly interested in the company’s financial health and sustaining its long-term growth and profitability. As such, KPIs play a crucial role in driving financial performance and aligning operational activities and efforts with strategic goals and objectives, ensuring that everyone is working towards the same outcomes.
The linking and analysis of KPIs to the company’s financial results is not fully successful until information is effectively reported to the Executive Level Team (ELT), as actions and important decisions need to be made accordingly. Reporting KPIs to Executives in a frequent manner can serve as the basis for productive discussions and collaboration among Executives, as this encourages dialogue around strategic planning, performance trends, challenges and opportunities, and enables informed decision-making.

When communicating and reporting KPIs to the ELT, it is of great importance to determine clearly what each KPI measures and why it is important for the business. It is also crucial to consider setting clear communication and presentation goals where relevant and meaningful information is presented to ensure that Executives have a comprehensive understanding of the organization’s performance and strategic direction.

When presenting KPIs, it is best to present them in a visually concise, focused, appealing, and easy-to-understand format via charts, graphs, and dashboards to illustrate trends, comparisons, and key insights. Including contextual information and analysis alongside may allow Executives to better interpret the data accurately. When deemed necessary, identify key findings and insights derived from the KPIs and highlight actionable steps or strategic decisions that need to be taken based on the results. Monitoring progress against KPIs over time is fundamental to track changes in performance, evaluate the effectiveness of strategies implemented, and determine the need to adjust KPIs or initiatives as necessary.

CONCLUSION

In conclusion, in today’s dynamic marketplace, it is vital for any organization in search of fostering a holistic approach to performance management, to be able to translate operational efficiencies into financial successes. As such, aligning a company’s operational KPIs with its financial statements can be of great help in support of this goal. The benefits of linking operational KPIs to the income statement and the balance sheet can be substantial, as they can drive organizations to better understand the financial implications of their performance metrics and make better-informed decisions to drive revenue growth, improve profitability, and achieve strategic objectives.

As the saying goes, what gets measured gets improved. The same is the case with supply chain management. End to end supply chain operations related to planning, sourcing, production and logistics have specific metrics that help us improve processes and overall performance.

For every metric, there is a target number or goal that needs to be agreed upon. Then, each of those metrics is linked with a strategic objective or a key value driver (KVD), and is further supported by planned initiatives that help in achieving the target or goal.

Demand Planning Metrics

Demand planning metrics relate primarily to forecast error metrics. They include:

• Mean Absolute Percentage Error [MAPE]
• Weighted Absolute Percentage Error [WAPE]
• Mean Squared Error [MSE]
• Forecast Bias, and
• Forecast Value Added [FVA]

Of these, Forecast Bias and FVA are critical and should be tracked over the planning time horizon. Bias indicates the direction of the forecast errors and helps us understand whether we are under-forecasting or over-forecasting.

FVA is a measure of the value added during the forecasting cycle. It is measured by comparing forecast accuracy before and after each activity in the demand forecasting and planning process to determine if that activity improved the accuracy of the demand forecast. Tracking FVA during the forecasting cycle enables us to find the right mix of quantitative and qualitative approaches that result in fewer errors.

Supply Planning Metrics

Supply planning metrics are related to sourcing and production. They include:

• Order Confirmation to Delivery Lead Time
• OTIF % [On Time in Full]
• Material Availability for Monthly Production Schedule
• % Rejections and % Returns
• Spend Analysis [By Product Mix and Supplier]
• Direct and Indirect Category Analysis and
• Supplier Performance Index

These metrics allow us to understand supplier performance and to perform spend analysis.

• Production/Manufacturing metrics include:
• Daily schedule adherence
• Production loss
• SKUs with excess production
• SKUs with shortage in production
• Line Utilization %,
• OEE% (Overall Equipment Effectiveness)

It is essential to segregate the metrics into daily, weekly, monthly and quarterly buckets. However, the most crucial performance indicator is OEE % as it gives a snapshot of equipment productivity, product quality, utilization and speed.

S&OP Metrics

During S&OP meetings, the appropriate metrics are reviewed and discussed with the respective functional teams during each phase or step of the S&OP cycle.

Logistics & Distribution Metrics

This section covers metrics related to warehousing, transportation and distribution operations.

Warehousing metrics

•  Dock to Stock Cycle Time
• Inventory Accuracy %
• Operating Costs [Budget vs Actual]
• Detention Costs
• MHE Utilization %
•  Process Productivity Metrics
• Order-Delivery Cycle Time and
• Environment-Health-Safety Metrics

The metrics that are critical and need extra attention are the ones related to inventory management and customer order management. Accurate inventory management within the warehouse is critical to ensure correct order picking, packing, staging and loading operations. End to end order management from order receipt to fulfilment to proof of delivery are central to warehouse operations.

Transportation metrics

• Truck Placement Reliability
• Loading Time
• Unloading Time
• Transportation Lead Time
• Operating Costs
• Vehicle Capacity Utilization %

In the case of transportation, vehicle arrivals at the appointed times are critical for timely onward operations. Monitoring transportation lead times are also essential, especially for last mile deliveries.

Distribution metrics

• Despatch Schedule Adherence
• OTIF [On Time-In Full] %
• Customer Order Confirmation to Delivery Lead Time
• Product Returns %
• Customer Satisfaction Score

As an extension, customer satisfaction and product returns metrics need to be measured and monitored. Most 3PL and 4PL service providers invest in processes and systems related to customer issues. Order fill rates and customer satisfaction metrics need to be focused upon and prioritized.

Other Relevant Metrics

The advent of e-commerce and quick commerce has resulted in ever-increasing reverse logistics and material flows. Therefore, metrics related to reverse flows are as important as forward flows. Also, macro-economic factors, geopolitics, climate events, trade flows, fluctuating commodity prices, etc. have necessitated the measurement and monitoring of resiliency metrics.

Reverse logistics metrics

• Logistics Costs and Product Returns as a % of Sales

The challenge here is to minimize or reduce product returns and therefore reverse logistics costs. Just like the forward supply chain, designing the reverse supply chain is essential in ensuring that returned products can be inspected, segregated and reused, repurposed or refurbished depending on the type of product

Resiliency metrics

• Time to Survive (TTS)
• Time to Recover (TTR)
• Financial Impact (Sales and Profitability)

Resiliency metrics became extremely important for supply chains following the COVID-19 pandemic. Companies soon realized how every supply node in the chain could affect downstream activities due to material shortages and supply delays.

For firms that had a global network of suppliers, the ‘Time to Recover’ took several weeks and months. Moreover, supply related constraints had an adverse impact on demand that eventually affected sales and profitability numbers. Firms have started to invest in tools and systems that enable ‘what-if’ scenario analysis – what the financial impact might be in case a supply node is unable to perform as planned.

Linking Operational & Financial Metrics

The various operational metrics and performance indicators listed above have a direct or indirect impact on business performance – more specifically, financial performance. Therefore, it is essential to establish a link between operational metrics and financial metrics.

The key financial metrics to focus on include:

• Cash to Cash Cycle Gross and Net Working Capital
• Inventory Turnover Ratio
• Gross Margin and Net Margin %
• Return on Assets (ROA)
• Return on Capital Employed (ROCE)

Cash to Cash Cycle, Gross and Net Working Capital and Inventory Turnover Ratio are the three metrics that should be on the radar of the supply chain management team. All the demand, supply and distribution plans and their execution have a direct bearing on the cash conversion cycle and working capital levels. ROA and ROCE are impacted by supply chain operating costs. Any reduction or saving in procurement costs, inventory handling costs, production and logistics costs shall have a positive impact on ROA and ROCE. The operational metrics can be dovetailed into a relevant business/financial scorecard every month, quarter, and financial year.

People, Processes & Systems

Finally, for any performance management philosophy to work, there is a need to put in place the right team with suitable skills and understanding of the supply chain domain in addition to devising and deploying the appropriate processes and procedures for data management (data recording, reporting, monitoring and analysis) and selecting the right tools, software and systems for reporting, analysis and visual management.

Visual management tools such as dashboards, functional control towers and related systems could go a long way in monitoring key operational metrics on a real time basis to enable course correction and development of corrective and preventive action plans going forward.

To get up to speed with the fundamentals of S&OP and IBP, join IBF for our 2- or 3-day Boot Camp in Miami, from Feb 6-8. You’ll receive training in best practices from leading experts, designed to make these processes a reality in your organization. Super Early Bird Pricing is open now. Details and registration.

I recently watched a Ted Talk by Professor Tali Sharot, a specialist researcher in Experimental Psychology at University College London. She described how optimism bias is rooted in our brains to the point where it’s an evolutionary trait.

Human brains integrate positive evidence more efficiently and faithfully than we do negative evidence. Our tendency is to overestimate the likelihood of experiencing good events in our lives. In short, we are more optimistic than realistic, and we are oblivious to the fact. 80% of the population display an optimism bias to some degree, so it comes to no surprise that this translates into the business environment.

This got me thinking about optimism bias in the world of product forecasting – how we adapt our behaviour, especially regarding new or recently launched products where there is a significant judgemental contribution. New product launches typically show a forecast accuracy of 40%-55% (after the first year of launch) and forecasts are heavily reliant on the judgement of stakeholders.

The optimism bias challenge is so prevalent in the real world that the UK Government’s Treasury guidance now includes a comprehensive section on correcting for it. A real-life example is the cost of hosting the Olympic Games which, since 1976, is over forecast by an average of 200%. If future bidders wanted to safeguard against this bias, they should bear this in mind.

Tackling optimism bias is therefor crucial. Research done by Professor Sharot shows that being aware of the bias does not shatter the illusion. This is key to our understanding as having a bias KPI in forecasting is not enough for it to go away- though it is certainly a step in the right direction. It shows us the magnitude of our bias but measuring alone is not enough to eliminate the human and organisational behaviours that drive it.

To add to the above, let’s also consider that there are two types of optimism. One is closely related to belief in our success and essentially backing ourselves. Studies consistently show that this type of optimism is valuable as it leads to success in academia, sports, and politics. For a new product forecast, we are backing ourselves and our team’s ability to outpace the competition and all our assumptions being right as we are personally invested in its success. This type of optimism is in fact necessary for stakeholder buy-in and team motivation, even though the reality is that not all our efforts will bear 100% commensurate results; some might deliver higher results while others will go the opposite way.

The second and more alarming type is blind unrealistic optimism, i.e. overconfidence in our assumptions and our ability to deliver. Overconfidence is “the most significant of the cognitive biases” for new launches according to research by the Tuck School of Business at Dartmouth. It leads to the setting of unachievable goals which initially might look tempting but, once the hoped-for results do not transpire, can lead to a demotivated workforce underperforming. This is often manifest by overlooking the basic facts and fundamentals and not planning for “What if” alternative scenarios. This is where mature forecasting and S&OP processes can play a vital role as they define risks and opportunities, drive scenario planning, and encourage get stakeholders from Commercial, Marketing, Finance, and Supply Chain on the same page.

From my experience, here are some of the steps that I have seen work best when it comes to ensuring that a forecast reflects all the opportunities without being unrealistically optimistic.

1. Promote Diversity of Thought

The question here is, who has input into your forecasts? We should be wary if only one team is influencing the forecast as this could lead to homogenous personality types dominating the inputs who will inevitably have blind spots. It is common to see teams who are so invested in their new product that they discount some basic facts around its launch assumptions. The more diverse the teams who have a say in the process, the more likely we will have a robust set of assumptions which have been pressure tested. We must ask the question: Does our culture actively encourage individuals to speak up and have a say, especially if they have a different opinion?

2. Document the Assumptions That Underpin the Forecast

On occasions you may hear from a colleague who has been intimately involved with new products saying, “I feel this will perform better than the competition or better than we previously thought”. But as demand and S&OP managers, we need to understand the assumptions behind the “feel” factor and what is driving this. Feelings don’t have numbers, however assumptions driving price, competitive intelligence, and market share can be analysed and documented to be shared with stakeholders. This is where we need to keep investing in soft skill training for demand and S&OP managers.

3. Do Scenario Planning

We live in a world where our decisions are influenced by our environment and work culture. There might be pressure to hit growth P&L targets, internal politics, or an individual is overriding everyone else. In new products, it is important to acknowledge we will always have the “known knowns vs known unknowns” in our assumptions. Setting up a Base vs Ambitious case scenario opens this debate at the S&OP table in a constructive way and forces us to break down what we know and what we are less confident about. We can prepare and execute both plans and, importantly, know what indicators will show which trajectory we are on. Base vs Ambitious do not have to conflict with each other. Having this discussion early sets us up for decisions on safety stock, late-stage customisation, inventory order points, and risk of write off, among other parameters.

4. Peer Review your New Product Forecast

One of the most effective S&OP practices I saw practiced in my FMCG career was cross-checking of assumptions by an internal, non-aligned stakeholder. For example, in Pharma this could be the commercial lead of Antibiotics stress testing a Respiratory channel forecast. In retail, this could be somebody from Health and Beauty looking at Clothing channel forecast assumptions. If this is done in an open environment and constructively, it produces a robust discussion that ultimately ensures the strongest forecast possible. This works best where the leadership culture of the company is open to taking feedback from the broader organisation and doesn’t work in silos.

5. Learn From Historical Launches

What caused you to be off your forecast last time you launched? Which assumptions turned out to be incorrect? This might sound like a fundamental building block of the process, but it’s important to review your recently launched products on a 3 and 6 month rolling basis (shorter in FMCG) and have a learning feedback loop. Documenting this early on in a central repository is helpful. Often after-action reviews are in presentations that get lost over time as people move to new roles, thereby losing the institutional knowledge gained along with it.

Summary

To have success in our launches, we must believe we are equipped to execute every opportunity without becoming overconfident. A new product launch that has its forecast backed up by a robustly evaluated plan that considers all eventualities will have a significantly better chance of being a resounding win.

Peter Drucker’s famous axiom “You can’t improve what you don’t measure” is particularly relevant to business forecasting. As Demand Planners, we want to measure our forecast performance so we can iterate and improve. Here I present an Excel-based Forecast Performance Tracker (free download available below) that you can use for your own error measurement.

There are  various methods and metrics to track and assess Forecast performance. A few of the most widely-used metrics are MAPE, WMAPE, MAD, MSE, RMSE, BIAS, Tracking Signal, as well as Michael Gilliland’s FVA (Forecast Value Added). Demand Planning teams monitor and report the forecast performance. When tracking forecast error through such metrics, it is essential to know why the error has occurred so the root cause can be addressed. There will be always be a certain amount of innate volatility and variability in forecasts. And, since the forecast is validated by human interference and judgements, bias is always present to some degree. 

Having an understanding of the error enables us to make decisions that will reduce it. Forecast error can be problematic for organizations – not only within supply chain/operations, but at an enterprise level. Though the steps taken based on the understanding of forecast errors are reactive, we can use those steps to reduce future errors. 

Forecast error simply defined is the difference between the actual demand (sales) and forecasted demand. Forecast Error = (Forecast – Actual) / Actual. Root Cause Analysis (RCA) can be split into 3 classifications: Over Forecasting, Product Unavailability and Under forecasting. The following table (Table 1) gives an insight into these  3 RCA classifications. 

Figure 1 | Root Cause Analysis Classification Model

The RCA Classification model above gives details our 3 classifications of Over, Under and Product Unavailability. The framework also gives details about negative or positive bias.  Importantly, it also displays a few of the potential impacts on the business. There is also one more factor we should be aware of that isn’t included in the table – Random Variation. In cases of Random Variation, the error generally corrects itself. 

Model To Track Root Cause Analysis Of Forecast Error

Over forecasting and under forecasting are widely discussed in the demand planning literature. However, I haven’t seen much discussion about product unavailability. An Excel-based forecasting KPI tracker is prepared (see a snapshot below).

[CLICK TO DOWNLOAD THE FORECAST TRACKER]

The most important elements are Forecast, Actual Sales, and Inventory (closing) for the given forecasting period (month, week, etc.). For simplicity, we are using 2 products (P1, P2) and 3 locations (L1, L2 and L3). The forecasting horizon is monthly, from January to April. Other details like Sales Representative, Product segment, and Categories can be added as per your business requirements. The purpose is to monitor forecasting performance by product and location on a monthly basis. 

You’ll also see the different error metrics: Error, Absolute Error, MAPE/WMAPE, Bias, Over Forecasting, Under Forecasting and Product unavailability. 

Screenshot of forecast tracker

Cont.

In this tracker, when you add the monthly forecast, actuals, and inventory data, the rest of the report updates accordingly. All the data analytics are managed in Excel with formulas, pivot tables, and charts. 

Forecasting Performance Dashboard 

The model contains an interactive dashboard which, at the end of the month, can be used to share forecast error in demand planning/S&OP meetings sessions as a standard report. The dashboard present the data via effective visualizations that depict the narrative behind key performance indicators, including key insights and recommendations on a single screen. 

The most important component of the dashboard is the key insights and recommendations. Going into any meetings where the dashboard is used, Demand Planners should have a good understanding of the major forecast errors and be ready to facilitate discussion surrounding actionable steps to remedy the causes. The aim is for senior management to make informed decisions. 

Below (figure 2) you can see the dashboard. The key features are the MAPE monthly trend, and top locations and products with highest MAPE for the month. For example, location L1 is experiencing error from under forecasting and therefore needs to be addressed in the meeting to identify what can be done to remedy it. Location L2 is facing under forecasting. To a certain extent this under forecasting is correlated to product unavailability since sales tried to compensate for the forecast target with available and on-demand products. 

Figure 2 | Snapshot of forecast tracker dashboard

Cont.

Benefits Gained From Forecasting Root Cause Analysis

As Arthur C. Clarke said, “ I don’t pretend we have all the answers. But the questions are  certainly worth thinking about.” This methodology enables exactly that – allowing you to measure forecast error and discuss root causes in a simple yet effective way. With insight into root causes, you can optimize your supply responses better and shape demand accordingly. Improved forecast accuracy will naturally follow.

 Key Takeaways 

1 – Demand Planners should demonstrate strategic value by bringing key insights and recommendations to facilitate informed decision-making.

2 – The purpose of such models is not to highlight ‘WHO’ (any function/role areas) but to  effectively address the ‘WHAT’ (cause for over or under forecasting). 

3 – Art is an important trait required for Demand Planners. They should convey the key  insights, and just the data. 

4 – Demand and supply variability is great these days, so be aware that forecast error improvement has a limit as we have no control over external factors impacting demand.

5 – Estimating all the components of error from the demand history is not possible (or even appropriate). Uncertainty is intrinsic. 

6 – Demand Planners should persistently develop data analytics skills with a clear approach to storytelling instead of only providing reports based on convoluted mathematical formulas. 

7 – Emphasis on forecast accuracy numbers will result in bias. Hence, the focus should be on providing key highlights to the Management team. The most consuming part of the reports is the insights and recommendations sections which enable businesses to take better decisions. 

8 – As mentioned in my previous blog, Segmentation Framework For Analyzing Causal Demand Factors, Forecast Accuracy is not the goal but a means toward the larger goals of the enterprise. 

Do you find this model useful? Is there any further enhancement that can be done? I am  open to hearing from you. 

Do you want to understand the logic behind this forecasting performance tracker and  dashboard in Excel? Connect me for a session. I will be happy to take you through the tracker and dashboard. 

Connect with Manas on LinkedIn and follow him on Medium.  

For more demand planning insight, join us at IBF’s Global S&OP & IBP Best Practices Conference in Chicago from June 15-17. You’ll learn the ingredients of effective planning, whether you’re just getting started or are finetuning an existing process. Early Bird Pricing now open – more details here.

S&OP processes always include a demand review step. There are plenty of articles, webinars, and blogs about how to set up and optimize a Demand Review, covering such topics as the best metrics to use, which departments to include, and whether to use a unconstrained or constrained demand forecast.

These details are important, but there is something often overlooked in our attempts to build a robust Demand Review. We often struggle to speak a “language” that is understood by those outside Demand Planning.  By failing to speak a universal language we our peers can undervalue our efforts and fail to see how what we do  relates to their work.  As a result, we struggle to keep Sales and Marketing partners engaged and, subsequently, we don’t get the necessary insights.

Details are important, but the priority for our Demand Review meetings is cover four key areas. Let’s look at the cornerstones of a healthy Demand Reviews and identify what value should be captured from each of them, including the one even experienced Demand Planners miss that is key to tying our demand plans to revenue.

1 – Understand The Assumptions Behind Past Performance

To start, we need to understand the value of our past assumptions. These assumptions include understanding our demand history, quality of the data, the models we use to create statistical forecasts, and the adjustments we made to accommodate business changes. We can’t possibly build trust in the forecast we are using if we don’t take the time to measure and track past performance. It’s why we have so many discussions around which metric is best, which level to measure, and how to apply formulas.

“Too much time is spent reporting performance as if this is the purpose of the meeting”

Understanding past performance is necessary, but frequently too much time is spent reporting performance as if this is the purpose of the meeting. Capture it, measure it, discuss it but don’t spend all your time on it. It is equally important to take the time to choose the right metrics. While a single metric can certainly tell you something valuable about the forecast, a singular metric limits understanding and often generates misconceptions about the forecast. Using multiple (compatible) metrics improves daily decisions when using the forecast.

This article isn’t to debate which measure is best, which formula to use or how to apply them; it’s a reminder to consider metrics beyond just looking at accuracy and bias. There are plenty of great metrics that you should add like FVA, Forecastability, tracking signals and waterfalls, forecast hit/miss, and even simple absolute error in dollars. Whatever combination you choose, explain the metric in terms of what they measure and what they tell you about the data. Advanced planners will take the time to link results to actionable behaviors in other areas.

“Discuss past assumptions and which ones were most value-added”

For example, forecast performance can explain poor service levels, tendency to build excess inventory (bias), or even why operational efficiency is low (poor product mix measured by accuracy calculated at lower levels). Measurements like FVA can tell us which assumptions, forecast or whatever we want to measure provided a better predictor of actual demand. Discuss past assumptions and which ones were most value-added.

2 – Assess The Current Demand Factors

Secondly, we need to understand our current demand situation. The Demand Review should also highlight our current immediate forecast and help us to understand how we are trending in actual performance. While this isn’t the ultimate focus of our meeting, it is something we should discuss. Are we waiting on orders or are orders ahead of plan? Is there a risk of creating future shortages?

Reviewing and discussing the current short term demand picture can also help us identify unplanned and previously uncommunicated events. Finding these events while they are happening — while less than ideal — can help us mitigate risks and drive better customer service in the future than if we were to find them when reviewing our past assumption performance.

Of course, it would be better to be told of these events in advance so they can be properly planned, but let’s face it — for every event or planned promotion there is at least as many and usually more that happen on the fly without any advance notification.

3 – Understand Future Demand Factors

Our Demand Review should help us understand the future. Most of our time should be spent on understanding the assumptions, events, new products, or sunsetting items loaded into the demand plan in the mid to long range period, as well as any risks or opportunities in that plan. Rather than thinking about the forecast for next week, next month or maybe two months out, we should be looking out to the next quarter or the next year in our forecasts. At a minimum, we should be looking at our max lead-time plus one period.

“The Demand Review creates a road map that all other departments in the organization will use to make decisions”

I’ve been in many Demand Reviews and have been guilty of leading reviews where most of our time was spent on looking at past performance. Sure, it helps us to understand something about our process, we must give at least equal attention to the future. The whole reason we’re building the demand plan is to have some understanding of what we can expect in the future and to share that knowledge with the rest of the organization. It’s to set a road map that all other departments in the organization will use to make decisions.

Understanding the assumptions and any risks or opportunities that exist provides context, credibility and builds trust in the forecast, which is critical if other functions are to use it. Thus, when using the demand plan, we can plan contingency strategies for things not in the plan instead of letting fate happen.

4 – Understand The Gap Between The Demand Plan & Financial Plans 

One last part often missing from demand reviews is a gap analysis between the demand plan and the operating budgets or financial plans. We work as a team towards a single plan, but those forecasts may look different depending on how they are translated such as the difference between the demand plan, financial plans, and the replenishment plan. As the demand plan rolls through the S&OP process into the supply review, disconnects between what we sell and what we can supply are identified and addressed. Unfortunately, we often overlook the need to compare the demand plan to the latest operating budget or financial plan.

It’s not just about comparing to the original plan from the beginning of the year — that will no doubt change — but instead understanding the gaps between the current financial plan and the demand plan. Are we planning for revenue above what’s in the demand plan? If so, what is being done to ensure the right product (or any product) will be available to sell to meet the revenue expectations? If revenue expectations are below the demand plan, do we understand why, and the potential excess inventory we are building? Communicating and understanding gaps improves opportunities to meet corporate initiatives.

Demand review are a critical step in any S&OP process.  As Demand Planners, we often get buried in a lot of details with large charts and files, talking about metrics that many of our sales and marketing partners may not understand or find valuable. Taking the time to communicate the right metrics and refocus efforts on the parts that matter most will result in a more productive demand review meeting.  It’s not about who is the most accurate, but rather identifying the data that provides the most value and capturing the insights needed to better run our business.

Not all items are created equal. Not all customers are not created equal. We know this to be self-evident. If this is true, why do we assume our accuracy should be the same for all items and customers?

As Demand Planners, it’s not enough to just forecast; we need to understand the underlining forecastability for each item. Forecastability may not be in Webster’s dictionary, but it should be a word we are all familiar with in demand planning. It’s how we measure the underlying uncertainty of a particular item so we can know what kind of accuracy we can expect from it, and what resources are worth allocating to a particular item or category.

To get some insight on this important topic, I invited Sujit Singh, CEO of Arkieva, to discuss this topic on the IBF On Demand Podcast. The following is based on that conversation.

Are All Items Forecastable?

If something has sold only 1 time, then we can agree it is impossible to forecast. Similarly, items with a long tail are not always possible to forecast. Items whose demand is impacted by external variables may not be forecastable either.

Any item that is stable with sufficient data points is forecastable; we can apply a range of techniques to generate a forecast, and in that sense it is forecastable. But the proof of the pudding is in the accuracy – regardless of the data points available or the techniques used to generate the forecast, if the resulting accuracy isn’t sufficient to help plan the business, it is not forecastable.

Items Are Getting Less Forecastable

We have all heard about long tail demand where demand is getting divided into more and more products whereby the portion of demand that is fundamentally unforecastable has increased. Sujit says that with this in mind, we can expand the definition of ‘forecastable’ by generating a forecast range (instead of a point forecast) and as long as we’re inside the range, the forecast is ‘accurate’ and therefore forecastable. In so doing, we can make these tricky-to-forecast long tail items more forecastable.

Reducing Forecast Error During The Pandemic

Given the current demand disruption caused by COVID-19, forecast accuracy is inevitably lower than that we which might have enjoyed prior to the pandemic. Should we increase our tolerance for forecast error? Sujit says if we give more weight to recent observations, isolate certain history and identify certain factors impacting demand, we can still get decent outputs from our time series. Of course, when demand assumptions change, forecast engines aren’t aware. A forecasting system doesn’t know a plant closed down, but your sales team will. With the right information we can then update the models and maintain some degree of accuracy.

Methodologies To Determine If Items Are Forecastable

Error is the main metric to identify which of your times are forecastable, but Sujit recommends another simple (yet useful) metric – Coefficient of Variation. The idea is we calculate the standard deviation of a time series and divide by the mean. Very often, the cutoff point forecasters use for forecastability is a CoV of 0.5 (the lower the number, the better). It’s effective but not a perfect measure of forecastability.

We can also use intermittence i.e., gaps between observations. Let’s say we are looking at data in monthly buckets with a sale in month 1, nothing in months 2 and 3, and a sale in month 4. Calculating the average delta between the non-0 sales. If you are more than 1.2, your series is considered highly intermittent and therefore difficult to forecast. You could still reach some forecast accuracy using specialized methods like Croston’s model, but it’s a challenge.

What Do We Do With Problematic Tail Items?

We all have items we struggle to forecast and would rather forget about. In such cases, once we’ve exhausted other methodologies like getting qualitative inputs from sales, Sujit recommends grouping these items and forecasting at higher levels of aggregation so the individual items ‘inherit’ some of the properties from the top level, thereby making them more forecastable. Let’s say you have 7 products, each of which are unforecastable individually and share some commonalities. Forecast at the group level then disaggregate, applying the weights to each item.

“Ah, my forecast accuracy was bad last month because marketing added a promotion too late…”

 “My forecast accuracy is fantastic; I achieved 70% for month 3!”

How many times have you heard one of the above from your Demand Planners? And in both situations what will be your reaction? I’m guessing in the first scenario you might be sympathetic towards the planner, and in second you might be excited for them!

If you find yourself in the first scenario, do not beat yourself up – it’s not the end of the world. It happens. While in second scenario, if you manage 70% accuracy that’s great!

Why Do We Create Forecasts?

But have you ever wondered why we build a forecast? Is it only for the sake of having good forecast accuracy? Let’s dive into why we need forecast accuracy as one of our KPIs.

In my opinion, we need forecast accuracy as a KPI for 2 reasons:

1. We need to measure the result of our consensus forecast vs. what happens.
2. Once we find the measurement, we need it to be easy enough for other stakeholders to understand.

Thus, as result we have either MAE or MAPE (or WMAPE) to measure how good our consensus forecast is vs. reality. Those KPIs are commonly used to communicate the result to stakeholders to inform them how accurate we are, accompanied by forecast bias to explain our tendency for either over-forecasting or under-forecasting.

And since Demand Planners build the forecast and lead the consensus meeting which results in the validated forecast, naturally forecast accuracy becomes part of their KPIs (and in many cases, also their bonus). This explains why Demand Planners are always very sensitive when it comes to the topic of forecast accuracy.

For some companies, this accuracy is also part of the business stakeholders’ KPIs. I personally agree with this because it is a good thing to make everyone aware that their decisions will have an impact on the wider business.

The Ugly Side Effects Of Forecast Accuracy

Now, let’s look at what I call “the ugly side effects” of forecast accuracy. I have seen some cases where Planners are doing too much or being too rigid for the sake of forecast accuracy, or even being judged based on this KPI alone.

1. The forecast is adjusted to deliver accuracy or “I do not trust the statistical model…”

I have seen this happen too often; the statistical model is overwritten with massive adjustments in an attempt to achieve good accuracy! If you are a planner and you are still doing this, before you continue, ask yourself (please!) “Is the adjustment meaningful? How much time will I spend doing this, and what will be the percentage accuracy gain?”

For the majority of the scenarios, the adjustment is not that meaningful and you will not gain much from your accuracy, and you spend a lot of time doing those adjustments. Next time, before you do this, please think about these points.

And if you need to do this because you do not trust your forecasting tools, I suggest you spend time understanding how the forecast is derived rather than to continue overriding it.

2. “But We can’t bring the launch/promo forward, it will hurt my KPIs”

I believe you have seen this scenario. Demand Planners arguing with Sales or Marketing about shifting the launch promotion as it will hurt KPIs!

Demand Planners, again I understand your reaction. But whenever this scenario plays out in your meeting, rather than argue about the KPIs (trust me it is not an interesting thing to argue about), ask your business stakeholders the ‘why’ questions. Why do we need to bring forward the promo/launch? Why do we need to postpone it? By doing this, you will get their point of view and why they are proposing this course of action. And based on their answer, you can see if it is reasonable. What would be the risk/opportunity for us here? What are the consequences?

From my experience, when those scenarios occurred, they did have a valid reason such as a launch being delayed as marketing needs to rework media to support the launch. The current one perhaps did not gain good ratings with the test consumers, or bringing forward a specific campaign would really help.

You can accept their point of view as valid and support it on the basis that it helps the overall business, while explaining what the consequences are on KPIs and inventory. Alternatively you disagree that it’ll benefit the business, or that it cannot be decided right away and the GM’s approval might be needed.

Remember, always think bigger! Think in terms of impact to the business, not from a KPI perspective only. So, be a bit braver ask “why?” and, based on the answer, you can work back to ‘Can this be supported and what would be the consequences be?” and “What would this mean in term of risk/opportunity?”

3. “You must be a bad planner… I can tell by looking at your forecast accuracy”

Worst one ever! Do not associate a bad result with a bad Planner. There could be lot of factors to explain why the forecast accuracy is bad, other than ‘bad Planner’. Do not be too quick to judge.

After all, how can we tell what good accuracy is? Benchmark vs. industry trend? Internal benchmarks? For me, to have those benchmarks is only half the picture. I will always look at forecastability to set my own expectations of what good looks like for a particular brand. For example, 50% for some brands might be all we can get.

From my own experience, I worked with one Planner for our make-up portfolio whose forecast accuracy on average was around 55%. Is she a bad planner? Oh, my goodness no! She is one of the best Planners I have ever met!

To explain why it is very tough to achieve above 55% accuracy, we did a variability calculation (not only for her portfolio, but for our total division). From there, we were able to understand based on variability alone what the best possible accuracy is for each brand in our total division. So, it can be 70% for some brands while for hers, 55% is the best we can get.

Back to her story – how did she compensate for her ‘low’ forecast accuracy? She did a great job in building her safety stock parameters and monitoring stock (managing excess/obsolete inventory) and worked closely with her major accounts which resulted in an improvement of her portfolio’s service level. And oh, the best part of the story? She managed to turn Marketing’s mindset from “Why even bother, our accuracy is bad anyway” to “Let’s do our forecast meeting so we can run our brand and serve our accounts”.

For me, as her manager, that was the highest compliment ever! When our partners are motivated to come to our forecast meetings and want to have conversations with us, we are doing something right and adding value to the business!

So, What Is The Point Of Our Forecasts?

If we take a step back, what are the points of building this forecast? The answers are surprisingly simple:

1. To facilitate decision making

Really, if you think about it, what is the main objective of all those forecast meetings you are having? It is for everyone to align on the forecast based on some scenarios we have all agreed upon. So that is the first impact of our forecast and then, based on that forecast, we will plan according to our supply needs that are translated into production planning (raw material planning too in factory side). And eventually, there is also an impact on logistics such as transportation planning and warehouse inbound activity.

All those decisions are taken from the forecast you build!

When I put it that way, I hope you now see that the time you spend in adjusting those forecasts for x% gain in accuracy might not be that significant. You might improve the accuracy, you might have improved your supply planning, but those results could be to a limited extent. We can spend that time doing something else like reviewing your safety stock or checking if your ordering strategy per month is the best for your warehouse inbound team. For example, if we receive the same item 4x by layer and if 4 layers make one pallet, is it possible to switch to receive it in one pallet?

If you address these points, you will have used your forecast in the best way imaginable; to facilitate decision making and enable efficient supply chain planning. So please remember that the forecast is built to serve this purpose.

2. To Ensure Customer Satisfaction

This is the next point – after all this planning, we want the customer to be happy, i.e to ensure the stock is there to serve our customers’ demand. So here is when other KPIs such as Service Level and On Time Fill Rate are used to measure our success as Demand Planners.

In conclusion, Demand Planners, the next time before you make an adjustment to your forecast or get in a heated argument with Sales or Marketing over a topic that could possibly hurt your accuracy please stop, pause for a while, and think “How does this fit in the bigger context? Can we afford this? What are the consequence and the risk/opportunity? Is it worth doing?”

Remember, forecast accuracy is just a way to measure and communicate the decision we made in our consensus forecast. We have a bigger purpose of doing forecasting: to enable decision making (for efficient supply chain planning too!) and to guarantee customers satisfaction. So, think bigger! Think Supply Chain !

Having worked in demand planning, S&OP and supply planning for several years, I have found that organizations often try to improve forecasts as a means to improve the overall supply chain. Indeed—looking at the pure math—improved forecast accuracy enables us to reduce end-to-end variability and optimize inventory and production efficiency. But the consistent focus on optimizing item accuracy takes our attention away from the real issues, i.e., our ability to manage variations and uncertainty to add value to our supply chain.

Regardless of our ability to improve accuracy by a certain number of points, we will still be short of 100%. The fact is we pay far less attention to driving business value than we do hunting for the next few points of improvement. In all the companies I have worked for, and the many companies I worked with during my years in consulting, I am yet to come across business processes that manage uncertainty in the forecast in an efficient way.

Using forecast accuracy as an input to the size of buffers needed and to assess the uncertainty in the plans is well established in many companies. However, in the operational execution, the acknowledgement of variability is absent. We still discuss the largest deviations between the forecast and actuals as opposed to deviations that are out of range. With that, our planning systems take us back to the target inventory level even though the consumption of our inventory may be perfectly in line with the expected variability, which may be different from forecasts. This kind of strategy causes a long sequence of changes all the way through the value chain, resulting in the well-documented bullwhip effect.

From a strictly forecasting perspective, measurement of forecast accuracy provides very little insight to improve it. Often, the way to improve accuracy is out of the hands of those working with the forecast. Accuracy is often affected by the way we incentivize our customers with different payment terms and shipping charges. I see three important steps that are needed to improve the way we manage uncertainty of the forecast.

Focus On Bias Rather Than On Accuracy

If we want to improve our forecasts, we should focus on forecast errors that are systemic such as forecast bias. Bias in a forecast is very harmful to the value chain. Measuring bias will help the business call out incorrect use of statistical models, where the sales history needs cleansing, and where qualitative “intelligence” that is manually added to the forecast is not intelligent enough. Bias is a main source of forecast error, which can be taken care of.

Using Segmentation For Allocating Forecasting Resources

Another way of dealing with accuracy is to assess the error against the natural variability in sales. A portfolio of items can be classified into segments based on their historical variability. Based on that, we can determine an expected level of error. The best way, therefore, is to chase errors that exceed our expected levels, and not ones that have large deviations. Say we have two items that are selling on average at a rate of 100 units per week. Item A has a historic variation of 30% and item B has a historic variation of 75%. Let’s say we sold 140 units of item A and 150 units of item B last week. Traditionally, we would investigate item B because the deviation of 50 units is larger than that of item A. Knowing that historically item A has a low variation and item B has a high variation, we should spend more time on understanding what happened to item A and investigate whether it needs re-forecasting, and not on item B which performed well within expectation.

Further, we should pay more attention to high value items, and less to others. As shown in Figure 1, we can expect higher deviations in products in segments 1 and 4, and lower deviations in segments 2 and 3. By looking at the volume (or revenue) as opposed to only deviations, we can identify the most important items to concentrate on. We don’t need to spend time on items in segment 4 (“exception only”) because they are of low importance to the business and have high natural deviations. Assessing deviations out of range provides an opportunity to refine our statistical models (as in the case of segment 2), as well as suggest how we can further improve the forecast of products in segment 1 by collaborating more with our stakeholders such as sales representatives.

Don’t Change Operational Planning If Deviations Are Within The Expected Range

Most planning systems work by trying to meet the target inventory. In that case, the plan is changed every time there is a change in actual demand compared to expected demand, no matter how small it is. In the network of inventory points and production sites, these little changes add up to much bigger changes. The cost associated with these changes throughout the value chain is in most cases not measured and accounted for.

Therefore, the best strategy is to have a system that stops re-planning whenever inventory is within acceptable range. In supply chain planning, it is often seen as an issue when actual inventory on hand is below target safety stock. Safety stock is meant to take care of uncertainty. If it does not, we are not using safety stock properly. Instead we are biasing our planning system, and unnecessarily putting pressure upstream. Every time inventory goes below a threshold, the system tries to replenish it. To avoid this, we should have a planning system like a forecast range. It should use inventory target ranges instead of fixed inventory targets. In other words, when the projected inventory is within certain limits, no change to the plan is necessary. When it exceeds the expected limit, as shown in Figure 2, the plan may have to be changed.

By using these three steps we can not only manage uncertainty more efficiently but also reduce cost. Some planning systems such as Kanban (reorder point planning) and pull-based decoupling points are good ways of reducing over reaction to variability in our operational planning. However, we do need a way to manage planning within ranges of forecast as well as inventory. In order to start reaping the rewards of managing uncertainty in this way, we must change how we think about variability and stop pointlessly chasing forecast accuracy.

This article was originally published in the Winter 2018/2019 issue of the Journal of Business Forecasting. Subscribe to get it delivered to your door quarterly, or become a member and get subscription to the journal plus discounted events, members only tutorials, access to the entire IBF knowledge library, and more.

Without KPIs, it is impossible to improve forecast accuracy. Here are 8 highly effective metrics that allow you to track your forecast performance, complete with their formulas.

Forecast Accuracy

This KPI is absolutely critical because the more accurate your forecasts, the more profit the company makes and the lower your operational costs. We choose a particular forecasting method because we think it will work reasonably well and generate promising forecasts but we must expect that there will be error in our forecasts. This error is a function of the time difference between the actual value (Dt) and the forecast value (Ft) for that period. It is measured as:

 Forecast Accuracy: 1 – [ABS (Dt – Ft) / Dt]

Where,

Dt: The actual observation or sales for period t

Ft: The forecast for period t

Our focus on this KPI is to provide insights about forecasting accuracy benchmarks for groups of SKUs rather than identifying the most appropriate forecasting methods. For example, achieving 70-80% forecast accuracy for a newly-launched and promotion-driven product would be a good considering we have no sales history to work from.

SKUs with medium forecastability (volatile, seasonal, and fast-moving SKUs) are not easy to forecast owing to seasonal factors like holidays and uncontrollable factors like weather and competitors’ promotions etc., their benchmark is not recommended to be less than 90-95%.

Tracking Signals

Tracking signals (TS) quantify bias in a forecast and help demand planners to understand whether the forecasting model works well or not. TS in each period is calculated:

 TS: (Dt- Ft) / ABS (Dt – Ft)

Where,

Dt: The actual observation or sales for period t

Ft: The forecast for period t

Once it is calculated, for each period, the numbers are added to calculate the overall TS. When a forecast, for instance, is generated by considering the last 24 observations, a forecast history totally void of bias will return a value of zero. The worst possible result would return either +24 (under-forecast) or -24 (over-forecast). Generally speaking such a forecast history returning a value greater than (+ 4.5) or less than (-4.5) would be considered out of control. Therefore, without considering the forecastability of SKUs, the benchmark of TS needs to be between (-4.5) and (4.5).

Bias

Bias, also known as Mean Forecast Error, is the tendency for forecast error to be persistent in one direction. The quickest way of improving forecast accuracy is to track bias. If the bias of the forecasting method is zero, it means that there is an absence of bias. Negative bias values reveal a tendency to over-forecast while positive values indicate a tendency to under-forecast. Over the period of 24 observations, if bias is greater than four (+4), forecast is considered to be biased towards under-forecasting. Likewise, if bias is less than minus four (- 4), it can be said that the forecast is biased towards over-forecasting. In the end, the aim of the planner is to minimize bias. The formula is as follows:

Bias:  [∑ (Dt – Ft)] / n

Where,

Dt: The actual observation or sales for period t

Ft: The forecast for period t

n: The number of forecast errors

Forecaster bias appears when forecast error is in one direction for all items, i.e they are consistently over- or under-forecasted. It is a subjective bias due to people to building unnecessary forecast safeguards like increasing the forecast to match sales targets or division goals.

By considering the forecastability level of SKUs, the bias of low forecastability SKUs bias can be between (-30) and (30). When it comes to medium forecastability SKUs, since their accuracy is expected to be between 90-95%, bias should not be less than (-10) nor greater than (+10). Regarding high forecastability SKUs, due to their moderate contribution to the total, bias is not expected to be less than (-20) or greater than (20). The less bias there is in a forecast, the better the forecast accuracy, which allows us to reduce inventory levels.

Mean Absolute Deviation (MAD)

MAD is a KPI that measures forecast accuracy by averaging the magnitudes of the forecast errors. It uses the absolute values of the forecast errors in order to avoid positive and negative values cancelling out when added up together. Its formula is as follows:

MAD: ∑ |Et| / n

Where,

Et: the forecast error for period t

n: The number of forecast errors

MAD does not have specific benchmark criteria to check the accuracy, but the smaller the MAD value, the higher the forecast accuracy. Comparing the MAD values of different forecasting methods reveals which method is most accurate.

Mean Square Error (MSE)

MSE evaluates forecast performance by averaging the squares of the forecast errors, removing all negative terms before the values are added up. The squares of the errors achieves the same outcome because we use the absolute values of the errors, as the square of a number will always result in a non-negative value. Its formula is as follows:

MSE: ∑(Et)² / n

Where,

Et: forecast error for period t

n: the number of forecast errors

Similar to MAD, MSE does not have a specific benchmark to check accuracy but the smaller value of MSE, the better forecast model, which means more accurate forecasts. The advantage of MSE is that it squares forecast errors, giving more weight to large forecast errors.

Mean Absolute Percentage Error (MAPE)

MAPE is expressed as a percentage of relative error. MAPE expresses each forecast error (Et) value as a % of the corresponding actual observation (Dt). Its formula is as follows:

MAPE: ∑ |Et / Dt |/n * 100

Where,

Dt: Actual observation or sales for period t

Et: the forecast error for period t

n: the number of forecast errors

Since the result of MAPE is expressed as a percentage, it is understood much more easily compared to other techniques. The advantage of MAPE is that it relates each forecast error to its actual observation. However, series that have a very high MAPE may distort the average MAPE. To avoid this problem, SMAPE is offered which is addressed below.

Symmetrical Mean Absolute Percentage Error (SMAPE)

SMAPE is an alternative to MAPE when having zero and near-zero observations. Low volume observations mostly cause high error rates and skew the overall error rate, which can be misleading. To address this problem, SMAPE come in handy. SMAPE has a lower bound of 0% and an upper bound of 200%. It does not treat over-forecast and under-forecast equally. Its formula is as follows:

SMAPE: 2/n * ∑ | (Ft – Dt) / (Ft + Dt)|

Where,

Dt: Actual observation or sales for period t

Ft: the forecast for period t

n: the number of forecast errors

Similar to other models, there is no specific benchmark criteria for SMAPE. The lower the SMAPE value, the more accurate the forecast.

Weighted Mean Absolute Percentage Error (WMAPE)

WMAPE is the improved version of MAPE. Whilst MAPE is a volume-weighted technique, WMAPE is more value-weighted. When generating forecasts for high value items at the category, brand, or business level, MAPE cancels plus and minus values. WMAPE, however, weights both forecast errors and actual observations (sales). When considered at the brand level, high value items will influence overall error because they are highly correlated with safety stock requirements and development of safety stock strategies. Its formula is as follows:

WMAPE: ∑(|Dt-Ft|) / ∑(Dt)

Where,

Dt: The actual observation for period t

Ft: the forecast for period t

Like other techniques, WMAPE does not have any specific benchmark. The smaller the WMAPE value, the more reliable the forecast.

The latest episode of IBF’s On Demand podcast is available to watch now.

Forecast Value Add (FVA) is a wonderful way to identify which inputs and activities increase your forecast accuracy and which decrease it. It can be a real game changer for your forecast accuracy and can identify bias from other functions.

Special guest Sara Park, Vice President, Exec S&OP, Forecasting & Supply Chain Planning at Coca-Cola, reveals how you can get started with FVA quickly and easily, and Eric Wilson CPF discuss why you should move away from MPE and MAPE to FVA.

Watch previous episodes:

• Episode 5: Is S&OP Dead?
• Episode 4: Predictive Analytics Implementation
• Episode 3: Segmentation & Clustering
• Episode 2: Forecasting Amid Coronavirus
• Episode 1: Will AI Take Our Jobs?