Modeling Long- and Short-Term Temporal Patterns

felix_wfelix_w Member Posts: 59  Maven
edited December 2018 in Help
Dear Rapidminer Community, 

I recently read about Multivariate Time Series Forecasting with LSTMs and tried to rebuild something similar in Rapidminer but I understood that this is only possible when using the Keras Extension? Is this correct? Or is there a way how to do this in Rapidminer without installing and setting up the Keras and Tensor environment?

In general, are there plans to integrate an operator set to quickly deal with LSTMs directly in Rapidminer Studio? Or is there already one and I totally missed it? ;) 

Best regards 
Felix

My sources: 
https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/
https://arxiv.org/pdf/1703.07015.pdf
Tagged:

Best Answer

Answers

  • hughesfleming68hughesfleming68 Member Posts: 214   Unicorn
    edited December 2018
    To be honest, the best way to use Keras in Rapidminer is to first learn how to use it in Python. It is not as straight forward as some of the other extensions. There are no out of the box LSTM operators in Rapidminer but you can do time series forecasting and I get good results from the built in operators. There is also the Deeplearning4j extension that is worth looking at. Learning to use the built in operators will also give you a point of reference to see if LSTM's work well on your data. I have had mixed results.

    I have bought Jason Brownlee's E-books on Deep learning and time series forecasting. They are helpful for getting up to speed with Keras.
    regards,
    Alex
    felix_w
  • felix_wfelix_w Member Posts: 59  Maven
    Hi Alex, 

    thank you very much for your reply! 

    Any chance that there will be "recurrent layers" implemented in the Deep Learning extension in the near future?

    Best regards
    Felix
  • felix_wfelix_w Member Posts: 59  Maven
    @pschlunder Wow, perfect :) :)

    Exactly what I needed! 

    Thank you! 
    mschmitzpschlunder
  • varunm1varunm1 Member Posts: 497   Unicorn
    edited December 2018
    @hughesfleming68 One big issue using keras in RM is that it doesn't give clear error details. I am working a lot on applying deep learning in python but when I tried to shift to RM with new Keras extension its throwing different errors. I even posted it in the Keras thread, unfortunately, no response from anyone. The same network, when applied in python, is giving the output. Also, is this keras extension tested with cross-validation (CV) operator? @pschlunder.

    Thanks,
    Varun
    Regards,
    Varun
  • hughesfleming68hughesfleming68 Member Posts: 214   Unicorn
    edited December 2018
    @varunm1. I had similar issues and just continued with Keras in Python.You might want to try the new Deeplearning extension that @pschlunder just released. It is based on Deeplearning4J. I have just started to go through the examples. It is probably a good idea to start a new thread just for Deeplearning as it is evolving very quickly.
    mschmitzvarunm1jczogalladbabrauskaite
  • varunm1varunm1 Member Posts: 497   Unicorn
    @jczogalla @hughesfleming68 @pschlunder Great. Thanks guys. Will work on the new release of Deep Learning extension.

    Regards,
    Varun
    Regards,
    Varun
  • luc_bartkowskiluc_bartkowski Member Posts: 46  Maven
    Hi, here's the how-to regarding RapidMiner Keras LSTM input-shape:

    1st: the problem domain: My input data is financial OHLC so 4 attributes/features.
    Additional assumption: Window size in the Window operator = 10, so 10 timesteps.

    I want to do classification, not prediction. Therefore I don't generate a label in the Windowing operator. If you do generate a label in the Window operator, or you use another amount of timesteps, horizon, etc. you have to adjust the parameters below accordingly.

    So in my application his results into an Keras operator input example set off 'n' records with 4*10 = 40 normal (windowed) attributes and a label. The Keras operator input shape is "(40,)" including parentheses and the comma.
    The Add Core Layer Reshape operator, that you add as the first operator within the Keras operator, should get a target shape of "(10,4)".
    This is the point where I suspect everybody goes wrong: Yes, we all use RNN but not as the FIRST operator in the Keras operator but it's the SECOND operator. Therefore output shape consist out of 2 parameters, not 3 (which generates an error).

    Hope this helps.
  • christos_karraschristos_karras Member Posts: 4 Contributor I
    Hi @luc_bartkowski,

    I've been trying to reproduce your solution but I can't make it work. I started from your initial XML sample, generated fake data to fit what the example expects, and then tried the modifications in your last post. I guess you did additional modifications to the original XML to make it work. Would you be able to provide your final solution?

    Thanks
Sign In or Register to comment.