How clean is your crystal ball: measuring forecast accuracy – Part 1

Demand Forecasting

In this blog I explain how to calculate forecast accuracy. Within the industry multiple approaches to calculate forecast accuracy exist, all serving different yet specific purposes. Just like any other metric, a one size fits all approach is dangerous and the KPI selected to be measured should fit with the overall objective of the organization.

As I wrote in my blog “driving efficiency through forecast accuracy”, the purpose of a KPI measurement should be clearly known, it is only then that its control and improvement will drive efficiency in the operations of the organization.

Supply Chain experts have therefore developed a toolbox of metrics that measure forecasting quality, these metrics generally come under the forecast accuracy umbrella.

Forecast bias

One of the most common metric is the Forecast Bias. Statistically speaking, as the name suggests, forecast bias is a measure of deviance from central behavior. It displays the tendency of an organization to consistently over or under forecast. It’s because of this property that this KPI is one of the most used ones. By having an unbiased forecast, sometimes called a 50/50 forecast, you can solve most of your big working capital and customer service issues.

  1. Formula variation

    1. Forecast bias is calculated by calculating the difference between forecast and actuals. The sign on the KPI tells you if actuals are over or under the forecast. For the KPI to be intuitive, under-forecasting should be denoted by a +ve sign, and over-forecasting by a -ve sign. This means that if you sold more than what you forecasted, the KPI would be a positive number and if you sold less than what you forecasting the KPI would be negative. In this case you would subtract forecast from actuals.

      Forecast Bias Formula - 1

      Forecast Bias Graph - 1

      However, some companies consider the term over-forecasting as +ve and under-forecasting as -ve, therefore yielding a positive number if you sold less than what you forecasted and a negative number if you sold more. To do this, you would subtract actuals from forecast.

      Forecast Bias Formula - 2 

      Forecast Bias Graph - 2

      Apart from the sign, there is no other difference in the result, However analysis should take this into consideration.

    2. The other element in the forecast bias formula is the divisor. The difference between Forecast and Actuals can either be divided by the forecast or the actuals, which would provide you with different results. This is significant especially if the difference between forecast and actuals is big.
      If you divide the result with the forecast when your forecast was much larger than the actuals, you are reducing the percentage impact of this under-sell comparative to if you divided the result by the actuals.
      Normally such extremes do not occur in reality, but what is important is the logic behind this. When you divide the difference by forecast, you are measuring the metric versus your forecast which should be your benchmark. However when you divide the difference by actuals, you are considering the actuals to be your benchmark versus which your forecast is measured.
      My personal opinion in this matter is to use forecast as a benchmark versus which the variation is measured because that is the figure the organization was striving to achieve.
  2. Units

    All KPIs are measured in some form of units. Since Forecast Bias is a ratio, it is expressed in percentages, however the source information (Forecasts and Actuals) are measured in some form of physical units, and all the different units that can be used have different interpretations.

    1. Countable & Measurable units:
      As we are measuring Sales and Forecasted sales, we normally use physical units like kilograms, Shipping Cases, Tonnes, or consumer units. The normal best practice is to use the same units in which the organization is invoicing at. This helps the forecaster to generate forecasts without any conversion factors. The other best practice is to always use “Countable” units like consumer units, boxes, cartons, pallets or bags, instead of kilograms or Tonnes. This is because a countable measure will always be more accurate than a measurable one because of it’s finite degree of accuracy. (you cannot sell 1/3 boxes but you can sell 0.3333333 killograms)
      Some companies tend to create a consumption unit of measure to reduce the impact of shipping case size variability in their portfolio.
    2. Value Bias:
      As forecasting normally feeds into a company’s S&OP cycle, a lot of organizations calculate forecast bias in the currency value. This approach provides them with an understanding of the value variation versus financial commitments. This helps in improved financial predictability and should be used along with “volume bias” to help operations.

Conclusion

Forecast bias is the key metric to establish a control mechanism on forecasting. When Forecast bias is kept in check with feedback into the forecasting process, many issues in financial shape and operational efficiency are reduced. This indicator can help the management eradicate deep rooted and fundamental issues within the organization giving them the ability to drive growth and allocate resources effectively.

Be the first to comment

Leave a Reply