i want to perform a comparison of forecasting performances for 5 methods: ARIMA, Generalized Linear Model, Linear Regression, Support Vector Machines and Neural Networks.
I use a sample dataset of Apple (AAPL) containing 137 consequtive trading days.
Since the ARIMA model is evaluated using the AIC score and i need all of them to be evaluated using Root Mean Squared Error (RMSE), i apply the ARIMA trainer of the first 127 days and then i ask the ARIMA forecast to predict over an horizon of 10 days, comparing the actual price with the forecast and calculating the RMSE.
For the other methods its much easier, since i train each model on the 127 days and apply the model on the last 10 days.
Note that i am using Grid search to find optimal parameters for all methods. The process runs perfectly but when i add the ANOVA operator to compare the performances, an error pop up. When i disable the ANOVA and have only T-test, everyting runs smoothly.
I attach the process (if you want to run this you have to disable the ANOVA operator!), the error and the data sample in a zip file.