Comments on: Introducing (E)Score: MAPE For Range Forecasts

By: eric wilson

eric wilson — Sat, 18 Aug 2018 13:43:20 +0000

Good question Robert: The probability may be calculated and like Thomas mentioned it could be derived from a confidence interval. I have seen some companies do this a few ways
Judgmental – apply an empirical range (i.e. +/- x number)
Naïve – use observed error of percent uncertainty and apply that +/- from a point forecast
Calculated – using COV or other models with a z(score) and assumption of normal distribution

Thomas: not sure fully understand your question on point verses multiple points but what you can do is the first part of the equation is RMSE. What can be done is no matter how many points it is the sum of the deviation from the actual divvied by the number of observations. With single point you are only summing one and dividing by 1 observation. If you have 10 you sum them all and divided by 10.

Like MAPE or any uncertainty metric it is better to use as an observation over time of performance than a single sample.

By: Thomas Amet

Thomas Amet — Sat, 18 Aug 2018 07:55:35 +0000

Thank you for the article. Very interesting approach.
However how to set the probability of the outcome to be in the forecast range ? Are we speaking about confidence interval probability ?

This e score can be applied to a single point of forecast but how would you use it to assess the overall performance of multiple points of forecasts ?

Thanks,

By: Robert Langenhuizen

Robert Langenhuizen — Fri, 17 Aug 2018 15:18:45 +0000

Nice article Eric and I think I would like to try it out! The only question I have is with the probability number. Where does a forecaster base him/herself on? Just calling out I’m for 90% certain it will fall in this range or does it come from a calculation?