Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

How to back-transform differentiated time series data?

manfyemanfye Member Posts: 1 Learner I
edited April 2020 in Help
I have done a ARIMA forecasting via a differentiated time series data with an ARIMA (1,1,0), but the data produced by rapidminer is the differentiated data, how do I back transformed it?
Tagged:

Answers

  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn
    I am not entirely sure what you mean by this--perhaps you can elaborate or attach an example?
    @tftemme might be able to help out he has done a lot of the development of the series operators.
    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Hi @manfye,

    You said : 
    I have done a ARIMA forecasting via a differentiated time series data with an ARIMA (1,1,0)
    Do you mean that you have applied a differentiation method yourself before fitting your data with the ARIMA (1,1,0) model ?
    If yes, what differentiation method have you performed on your initial time series ?
     - subtraction
     - logarithm
     - a combination of both ?

    I think it is possible to back-transform your predictions by using :  
     -  the "cumsum" (cumulative sum) function in a Python script (in case of subtraction) - I envisage to use Python because I'm not aware of an equivalent function of "cumsum" in Rapidminer.
     - the "exp" function (in case of logarithm)
     - and finally a combination of "cumsum" and "exp" function in the third case above.

    To conclude, as Brian said, can you give more detail about your process and your data and ideally share them ?

    Regards,

    Lionel










  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Hi again @manfye,

    I don't know why, but my intuition tells me that you used the subtraction differentiation method on your original time series.
    In this case, a priori you have obtained the predictions on the differentiated data by applying the ARIMA model.
    Thus you have obtained data like that : 


    Once you have these predictions on the differentiated data, you calculate the cumulative sum on this data and you add to this calculated time series, the first data point of your original time series - in other words, you integrate (from a mathematical point of view) your differentiated data -, and finally you retrieve the original time series with its forecast : 
     

    You can adapt the process in attached file to your own data. In my case, I used a time series called "Monthly milk production"
    available in RapidMiner repository.

    Hope this helps,

    Regards,

    Lionel

    PS : This process is using a Python script, thus, you first have to : 
     - install Python on your computer.
     - install the Python scripting extension from the MarketPlace.
     - set, in RapidMiner settings, the path where your Python.exe file is stored.


  • tftemmetftemme Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 164 RM Research
    Hi @manfye

    To add on the nice and detailed answer of @lionelderkrikor

    ARIMA in itself performs a differentiation before fitting the model to it (the "I" in ARIMA stands for "Integrated"). The number of differentiations applied is specified by the parameter d of the (p,d,q) parameter set. So in your case it was differentiating the input series 1 time and then fitting a p=1, q=0 ARMA model to it. And when applied to perform the forecast the, the ARMA model is used to produce the forecasted values which are then integrated again automatically. 
    So in short you don't need to differentiate your input time series by yourself.

    Another point. If you still want to integrate a differentiated time series (or any time series by the way), we will have a new Integrate operator in the upcoming 9.7.0 version of RapidMiner Studio. Besides cumulative sum, you can also apply left and right riemann sums and the trapezoidal rule to calculate the integral.

    Hopes this helps
    Best regards
    Fabian
  • prashant768prashant768 Member Posts: 6 Learner II
    Hi @tftemme, I doubt the ARIMA model in RM is doing the inverse transform after applying the differencing parameter. I have tried to perform the same in the data set and the forecasted values are getting too much out of scale. I have attatched the RM process for your reference. 
    I am really struggling to configure how to get back the forecasted values into the original scale as I do not want my students to use PYTHON because I am teaching them how to do analysis without any code
  • hmhsinghmhsing Member Posts: 29 Maven
    Hi @tftemme, I have a similar problem the forecast values are "explosive" when d is not zero. Hope this can be solved, thanks!  https://community.rapidminer.com/discussion/59825/arima-model-forecast-for-nonstationary-data 

  • rakhuuokrakhuuok Member Posts: 1 Learner I
    edited December 2022
    I don't know why, but my intuition tells me that you used the subtraction differentiation method on your original time series.  vidmate.app

  • salmawisokysalmawisoky Member Posts: 1 Learner I
    edited March 12
    Differencing is performed by subtracting the previous observation from the current observation. Inverting the process is required when a prediction must be converted back into the original scale. This process can be reversed by adding the observation at the prior time step to the difference value. 
Sign In or Register to comment.