Viewing results of run operators when an error occurred at the end

mansourmansour Member Posts: 26 Contributor II
edited June 2019 in Help
I have run heavy-duty rapidminer models for a few days, at the end I see there was an error with writing the Excel file. Do I have to restart all the processes again? Are there any ways which I can see the results of the modelings or run the process from the point it faced with an error?
Regards.
Mansour
Tagged:

Answers

  • varunm1varunm1 Moderator, Member Posts: 1,207 Unicorn
    Hello @mansour

    That's a good question, I do face this issue sometimes and it's really hectic to run the process again. Let us see if there is any viable solution for this rather than process rerun. I am not sure if there is anyway other than storing results before writing into excel file. 

    @mschmitz or @IngoRM any comments on this?

    Thanks
    Varun
    Regards,
    Varun
    https://www.varunmandalapu.com/

    Be Safe. Follow precautions and Maintain Social Distancing

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    yes I generally recommend storing parts of your process along the way for large runs. Standard practice.
  • mansourmansour Member Posts: 26 Contributor II
    Hi sgenzer
    I am storing the results of model performance into an excel file when the model faces an error; before it, there is no result to save.
  • varunm1varunm1 Moderator, Member Posts: 1,207 Unicorn
    Hello @mansour

    The storing informed here refers to the use of store operator, to save the results in rapidminer repository. 
    Regards,
    Varun
    https://www.varunmandalapu.com/

    Be Safe. Follow precautions and Maintain Social Distancing

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist
    edited June 2019
    Hi Guys,

    here is the bit longer answer:

    How does an operator work?

    Basically each operator has one method, to do it's thing (called doWork). Typically doWork methods are like this:
    1. Get Data From Ports
    2. Do Something With it
    3. Deliver result to output ports.

    So this means, that if we hit an error in (2), we cannot display much. The only way of doing this, would be to preserve the data when getting it from an input. That essentially means, that we need to do:

    1. Get Data from Ports
    1b. Store a security copy of the data
    2. Do something with it
    3. if it succeded, deliver result to output ports
    3b. Else, use the security copy for something

    Obviously this either takes a ton of more RAM or you need to store the backup to disc. That creates disc i/o and is thus also kind of expensive. Maybe our engineering team can do some magic by doing it in a seperate thread or so, but it will always cause some negative performance side effect.

    That explains why this feature is not (yet) in. But we discuss solutions for this internally.

    Best,
    Martin

    P.S Keep in mind, I'm a DS and not part of the RM Eng team. I might be mislead here.
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,993 RM Engineering
    Hi,

    I want to add one thing: If your process is running in the main Studio UI, you can also right-click a port of a finished operator and select to view the data from it. This even works if a later operator had an error. There is one caveat: If you're running low on memory, this temporary storage is purged. In that case, the data is not there anymore.

    Regards,
    Marco
Sign In or Register to comment.