Backup and retrieve modeling / Training status ?

Fred12Fred12 Member Posts: 344 Unicorn
edited November 2018 in Help

hi,

 

sometimes a training with grid optimization and different parameters for models (SVM, Tree, etc.) takes several days to weeks... but sometimes it happens that current is cut or something happens because of other problems... then I have to try out everything from beginning... now is it somehow possible to store the state of the machine while it is training the model with the parameters somehow intermediary? I mean like every 1 hour or so make a kind of "backup" of the training status, save it somewhere, and when it crashes, start from the backup and not start from square one?

 

is there any possibility to do that?

Best Answer

  • bhupendra_patilbhupendra_patil Administrator, Employee, Member Posts: 168 RM Data Scientist
    Solution Accepted

    Hi Fred,

     

    Please see the video below.

    If something like this may be useful for you. I have also attached the sample process/workflow here

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist

    Hey Fred,

     

    what abou this idea: You store a model every hour. If you need the parameters for another learner, you can use clone Parameters to transfer it.

    Some branch operator with date_now() and %-function should be fine to do the "every hour" thing

     

    ~Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • MCsMCs Member Posts: 8 Contributor II

    This "Save Intermediate Results" approach and solution was very helpful and came just in time for me, too

    Thanks @bhupendra_patil !

    Chill

     

  • Fred12Fred12 Member Posts: 344 Unicorn

    thanks, that's pretty cool :)

  • Fred12Fred12 Member Posts: 344 Unicorn

    I have an additional question to that...

    sometimes if I have too much parameters, its hard to save all the results and parameter models into a file, as its getting too much...

    is it somehow possible to do a saving like say every 10 minutes or every hour with some sort of timer or so? then save the model and results of the performance..?

Sign In or Register to comment.