Iterating, Macros, Automation

NoelNoel Member Posts: 82 Maven
edited October 2019 in Help
Hi All-

I've attached a process and excel data file. The process looks like this:

Inside those green boxes are two dates and assume the nonexistent Store procedure has a file name in it.

I need to accomplish a few things:
  1. I need to find a way to iterate over dates in the green boxes. (E.G. say I had 4 years of timeseries data... How could I automate the process of training the model on 24 months, applying the resulting model to one week for testing, advance one week, train on the next 24 months and then test on the next week, rinse and repeat?)
  2. What is the best way to store Apply Model output (i.e. predictions with confidences, etc.) and performance metrics (confusion matrix-type stuff) for each iteration such that one can go back and look at them all in aggregate. (E.G. How could I *cumulatively* compare the performance of 24mo training periods/test on one week for a year vs 36 mos or 18mo training periods?)
  3. I suspect there is a better way to cobble together a year's worth of weekly output than spewing out 52 excel sheets and stitching them together by hand... any suggestions?
Any sample processes would be most appreciated! Actual examples are most helpful.



  • Options
    lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Hi @Noel

    Interesting problem....

    I try to find a solution for the requirement 1.

    I converted the date to numerical (via Date to Numerical operator ..logic :) ) and then I embedded your 2 Filter Examples in 2 Loop operators. I try to iterate to create your training windows and your test windows but the output of the Filter Examples are empty ....
    for an unknown reason..!!
    It seems that there is bug inside the Filter Examples operators when condition class = expression is set 
    I missed something ....

    In attached file your modified process (don't worry, due to what I described above, the process raises an error...)



  • Options
    NoelNoel Member Posts: 82 Maven
    Wow, thanks for taking a crack at it, Lionel. I need to catch up on some much needed Zs, but I’ll have a tomorrow. Much appreciated!
  • Options
    NoelNoel Member Posts: 82 Maven
    @lionelderkrikor, et al-

    What if one created a schedule and stored it as a dataset. E.G.:

    Could one get those values into the loop (and other) operators in the process?

    Again, thanks.

Sign In or Register to comment.