How to improve forecasting

[noetsi]

I forecast monthly spending two years into the future. While our error is generally acceptable (5 percent or so a year out) I want to improve on that. I run exponential smoothing models (six of which I pick the best 3, best being defined by a MAPE from a hold out data set). I am trying to add an ARIMA models although that is experimental. I update my estimates once a month, we get new data only once a month.

No one at HQ, probably no one period, knows our spending process very well and there is no interest at HQ (other than by me) to do so. So applying expert judgement to the process is essentially impossible although I have tried. All we have right now is past data.

Any suggestion to improve this process, would be appreciated.

 

[Miner]

I think you are doing incredibly well to forecast two years with so little error. My understanding of ARIMA is that you should not use it to forecast more than 3 months. This is not due to a weakness in the method, but due to the inherent unpredictability of cyclical behavior.

 

[noetsi]

I think that is a generic problem miner with time series. I keep trying to come with new miracle ways to improve my methods. My theory is, more complex is better (which is not what the literature says actually for time series exponential smoothing is better in the M trials)

 

[Miner]

In general, I will use the simplest possible method that provides the results I need. It sounds like your company HQ is satisfied with the 5% error. Even if you got it down to 4%, all it will take is a management decision to launch a new program/product, or a pandemic to hit, and your forecast is no longer relevant.

 

[noetsi]

The pandemic has destroyed my models (they were working great until this month). The problem with the exponential models I use is that the disruption for even a few months could cause long term prediction issues. I can treat them as outliers I suppose and replace them in the time series with more reasonable values.

 

[Dason]

Depending on the methods you're using you can certainly model the shocks that were introduced

 

[noetsi]

Depending on the methods you're using you can certainly model the shocks that were introduced

Exponential smoothing does not model the shock in the same way ARIMA does with moving averages although it does model them. But I am not sure it would matter, because I don't think there is any way to be sure how long this shock will last, if the system will return to normal, and if so how long it will take. It is the same issue hays et el raised with their ARIMA models....

 

[critiasun]

In proc arima procedure, how to output MAPE(Mean Absolute Percent Error)

 

[noetsi]

I do MAPE in excel. I don't know if proc arima does a mape automatically.

 

[critiasun]

I do MAPE in excel. I don't know if proc arima does a mape automatically.

I inquired the SAS Help File, there is no MAPE Parameter option too. So I think there is no default option in proc arima until now.

 

[noetsi]

Its easy to do in excel and I do it all the time. You just transfer the data from SAS to excel.