Lora Cecere - Altimeter Group

Trading Places

The storyline is an old one. It was the theme of the 1983 American comedy titled Trading Places starring Dan Aykroyd and Eddie Murphy.  You may remember it; It is one of my favorite funny movies in which an upper class commodities broker and a homeless street hustler switch roles when they are unknowingly made part of an elaborate bet.

It is an ageless theme where someone less fortunate trades places with a more fortunate person.  As a child, I was enthralled as I saw it play out in Mark Twain’s Prince and the Pauper and Disney’s Parent Trap. While these are fictional stories, this week, I found a story where it happened in real life. Some of my favorite supply chain management leaders, organizations that I have worked with over the past seven years, had traded places in their organizational capabilities to forecast demand, and it was not a conscious choice.

Prelude

Before I tell the story, let me share a quick perspective on benchmarking demand metrics.  I have been working in this area for seven years.  It is one of the hardest area of the supply chain to benchmark.  I would like to take this opportunity to share my personal experience.

While companies eagerly want the data that benchmarking reports provide, benchmarking forecast accuracy is tricky.   Why is it so hard?  Let’s start with two major reasons:

• It’s Hard to get Apples to Apples.  It is a Fruit Basket. The first reason that makes it tough to benchmark forecast accuracy is that every company does it differently.  When doing this type of work, it is essential to have an “Apples to Apples” comparison.  To do this, you need to look closely at five variables:  frequency of  planning, granularity of the planning or does the organization use  monthly, weekly or daily planning, the construct of the data model, the input into the data model (E.g. shipments, orders, channel data), and the drivers of demand forecasting variance such as promotions, seasonal builds, etc. To get it right, the data must be scrubbed and normalized to ensure an “Apple to Apple” comparison.  As a result, companies should never accept data from self-reported sources.
• The Apple doesn’t fall far from the Tree. The second reason that benchmarking forecast accuracy can be difficult is the fact that the data can be hard to get.  To be useful, and since market conditions change, the data set needs to represent a like peer group from the same point in time.  Since many companies have multiple supply chains, and competitors tend to not want to share data directly with their competitor, getting the data is quite a feat.

Prologue

I ran into the, CEO of a major forecasting software developer earlier this week, and I was excited to find that he had just finished a project to benchmark demand data for consumer products companies to be deployed with his software solution.  Five of the companies were organizations that I had benchmarked in 2003 and worked with over the past five years.  While, neither he nor I can share the names of the companies, I would like to share my insights on their journey. It is truly a story of Trading Places. (In table 1, I have made up fictional names to hide the identity of the companies involved in the case study.)

The Story

While this story may not be as much fun as the original Trading Places movie, it is a real story where a focus on supply chain basics made a difference. In Table 1, I show the relative positions of the companies in the two analyses:

Table 1: Comparison of Five Consumer Products Companies Forecast Accuracy

Progress? For the group of companies that were benchmarked, the average monthly Mean Absolute Percentage Error (MAPE) for a one month lag was 31% + 12%.  Data eight years ago for the same companies was an average of 36% + 10% MAPE.  What was the result?  This group of consumer products leaders has gotten slightly; but not significantly better in demand forecasting.  They have weathered the storm of market changes that could have made the forecast much worse.  The industry has experienced major shocks including shorter product lifecycles, product proliferation, higher levels of promotions, changes in competitive behavior, and global expansion.

Better Math? Consistent with other industry studies over the last ten years (E.g. IBF, AMR Research, etc); the data from the present study shows that we have not made progress to improve forecasting & planning processes in leaps and bounds through the use of statistical models.  In the new benchmarking study, the use of statistical modeling software improved the forecast 3% on average (on a MAPE level with a 1 month lag) when compared to a naive forecast where this month’s volume planning is based on what was shipped last month.  In the top quartile of customers, the impact was 2X or a 6% improvement in MAPE.  A fact that was consistent in both studies showed that when the forecasting group reports to sales, the forecast bias is higher.

What is a 6% improvement in forecast accuracy worth?  Based on AMR Research correlations, a 6% forecast improvement could improve the perfect order by 10% and deliver a 10-15% reduction in inventory. Slow moving items on the tail of the supply chain are most greatly impacted by this.  Unfortunately, most companies let their supply chain tail whip them around.

It doesn’t just happen.  Basics matter. For me, the interesting story can be found beneath  the data. I am referring to the switch in position of the players over the course of the past eight years.  In this time period, the best in Class Company from 2003 became the worst performer and two lowest performers propelled themselves upward.  As I thought about why, and recounted my many experiences with these companies, several ideas came to mind:

• Moving down. The company that showed the worst performance in current benchmarking and the best performance in 2003 had a very high bias.  Why do you think this is the case?  The company made a decision shortly after the benchmarking in 2003 to have the forecasting group report through sales where there was a pervasive belief in the organization that if the company over-forecasted that sales would be higher.  This decision increased bias and cast a cloud over the forecasting & planning process.  The lack of a “true North” in the organization became a stumbling block to improving forecast accuracy.
• Moving up. The companies that moved up in the analysis, focused hard on the basics. This included efforts to clean data, frequently tune supply chain planning software, a strong corporate demand planning team that reports through supply chain and the use of the statistics.

Thoughts on tactical forecasting: While technology vendors like to brag that the use of their technology will make a difference in supply chain leadership, the data here is inconclusive to that point.  Instead, what made a difference in relative position was the process, data, and organizational reporting.  I know this may  not be the sexy stuff, but the basics matter.

Wrapping it Up

I commend this software developer for spending the energy and the manpower to benchmark their client base.  This type of commitment to ones client base differentiates and creates long-term relationships.  It is my hope that this type of analysis will be able to be part of continuous efforts for supply chain leaders.

I look forward to sharing these journeys and many other lessons that I have learned from my experience at IBF’s Demand Planning & Forecasting: Best Practices Conference w/ Demand Management Forum in Dallas, Texas USA this coming May 2011.

Please let me know your thoughts!

Lora Cecere,
Partner
Altimeter Group

Hear Lora Speak At:

Lora Cecere: Supply Chain Shaman

Go to any supply chain conference, and you will hear it.  Yes, the term collaboration is bandied about. It is over-used and often over-hyped in discussions largely without meaning.  So, what does it mean?  And, what happened to the supply chain collaboration initiatives of the 1990s?

Let’s start with the definition.  The greatest success in supply chain relationships is when true collaboration happens.  What does it look like? It is a when a sustainable win/win value proposition.   Six elements are required:  resources, skills, joint vision, leadership, a plan and aligned incentives.

The problem is that the so-called “collaborative programs” of the 1990s focused solely on process missing the mark on these six elements.  The tenants of VMI and CPFR were well-intended, but they fell short in building true collaborative relationships.  Let’s take a closer look.

Why did CPFR not gain wider adoption?

Many tout success, and many conference presentations expound on benefits; but back home at the office, the teams are confounded.  In the late 1990’s it was all the rage.  Yes, CPFR (Collaborative Planning Forecasting and Replenishment), over-hyped by many, has fallen short in delivering the promise

The results are clear.  After ten years of active projects, collaborative planning forecasting and replenishment failed to reach its promise for three reasons:

• Laborious. Just too much work for the benefit.  The added costs did not measure up to the benefit and the programs were not grounded in the six essential elements of collaboration.  Instead, it was a process implemented in the absence of the core elements of what drives collaborative relationships.
• Retail forecasts not up to the task. For CPFR to work, retail forecast accuracy needs to be high and with sufficient granularity to ensure analysis.  The dirty little secret with CPFR is that only three retailer forecasts—Best Buy, Food Lion and Wal-Mart—were up to the task.  In addition, the gap in retailer data for perpetual inventories and accurate on-hand data could not give the teams a good starting point.
• Lack of integration into Enterprise Systems. For most Advanced Planning System (APS)/Enterprise Resource Planning (ERP) deployments, there was no logical connection for the data.  As a result, it failed to make a systemic impact on supply chain excellence.

So, as a result, most CPFR initiatives became 20-year old pilot projects.  They were isolated—lacking integration into corporate demand planning architectures—and only as effective as the strength of the relationship and the quality of available data.

When does it make sense?

However, let’s not throw the baby out with the bathwater.  It would be incorrect to say that CPFR never makes sense. It was over-hyped and over-promised, and applied to situations where there was not a good fit.  So, you might be saying, where does it fit?   When a company has these five stars to align, CPFR can be used to reap great benefit:

• Significant channel presence. The account needs to be significant—at least 10% of the channel-for the investment to warrant the expense.  The greater the channel presence, the greater potential benefit. It must matter and make a difference.
• High Demand Volatility: CPFR makes more sense for products with short life cycles, seasonal patterns, strong dependence on weather, and in competitive categories.  It makes less of an impact for products that have stable demand.  Companies benefit from advance warning signals.
• Strong Retail Partnership. The data is clean, available and meaningful to both parties’ business objectives.  Both companies have strong planning skills and a passion for forecast accuracy.  It is tied and closely coupled to the business.
• The Tie to Replenishment can make a Difference: Many companies forget the “R” in CPFR.  If he advanced notification from forecast sensing can make a difference in improving replenishment and the other conditions can be satisfied, go for it!  However, not all replenishment cycles can be shifted in concert with the CPFR signals
• Right Stuff: It makes sense when there is demand architecture to support close coupling of the demand signal.  The architecture must allow integration at the account, ship-to level. 

What now?

The key is to be judicious.  CPFR has a place in driving supply chain excellence; just not the over-hyped promise of ten years ago.  Be more judicious.  Be more realistic.  Make smarter decisions.

See Lora Speak at IBF’s
First Ever:

Supply Chain Forecasting & Planning Conference: Asia