🎉 🎉 RAPIDMINER 9.10 IS OUT!!! 🎉🎉

Download the latest version helping analytics teams accelerate time-to-value for streaming and IIOT use cases.

CLICK HERE TO DOWNLOAD

Log generation in Rapidminer Studio

ZahraSadeghiboga92ZahraSadeghiboga92 Member Posts: 3 Newbie
I have seen the execution log generated by AI Hub which reports individual steps of execution of a process.

INFO: Started operator: Process
Jan 13, 2022 7:35:55 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
INFO: Started operator: Loop Files
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (0 sec)
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.logmanipulation.AddClassifierOperator doWork
INFO: Start: add classifier
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.logmanipulation.AddClassifierOperator doWork

However, I need more verbose logging which can be used to chain together the sequence of events that is taking place in a complex process.

At this point, the only solution seems to be modifying Rapidminer Studio source code (github.com/rapidminer/rapidminer-studio) to emit such a process log. Can you please point out to me wherein the source code does it make sense to perform this?

Thanks in advance,

Answers

  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,153  RM Data Scientist
    Hi,
    not sure if it helps, but the main process as a parameter called log verbosity. Maybe you just want to increase it?

    BR,
    Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,153  RM Data Scientist
    Oh, and you can write into this log using the Write to Console operator.
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • ZahraSadeghiboga92ZahraSadeghiboga92 Member Posts: 3 Newbie
    edited January 15
    Hello,

    I think that log level parameter only affects the operational log of the RapidMiner and not the execution log of the processes.

    To clarify, I should add, that the only information about an operator in the execution log is the start and end time. But there is no information that can help us chain together a sequence of related events when there is a parallel Loop Files operator in a process.

    For instance, the log will have several starts and end times for executing an operator without knowing which one is for which iteration of the loop.

    Inside "loop files"

    Moreover, I want to log the parameters that are passed to each operator.
    I have attached a sample of an existing execution log, and also what it should ideally look like:

    Actual execution log:
    INFO: Started operator : Process
    Jan 15, 2022 4:51:04 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
    INFO: Started operator : Loop Files
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec)
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec)
    Jan 15, 2022 4:51:05 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec)
    Jan 15, 2022 4:51:05 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishdOperator
    INFO: Finished operator : Loop Files
    Jan 15, 2022 4:51:05 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
    INFO: Finished operator : Process

    Expected execution log:
    INFO: Started operator : Process
    Jan 15, 2022 5:18:35 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
    INFO: Started operator : Loop Files
    Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log second_file.xes
    Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log first_file.xes
    Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec) first_file.xes
    Jan 15, 2022 5:18:38 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (3 sec) second_file.xes
    Jan 15, 2022 5:18:38 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
    INFO: Finished operator : Loop Files
    Jan 15, 2022 5:18:38 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
    INFO: Finished operator : Process

    My guess is that I will have to modify the source code of RapidMiner.
    Can you please point out to be the appropriate place in the source code where this can be done?


  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,153  RM Data Scientist
    Hi,
    why don't you use the normal Log operator for it? It can just log parameter and execution times of operators. Not sure why you  want to code something.

    Best,
    Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • ZahraSadeghiboga92ZahraSadeghiboga92 Member Posts: 3 Newbie
    Hi
    I want to get execution log from any existing workflow, as is, without modifying it.

    So for example, some else might have provided a large complex workflow, and I want to get execution log from it. modifying it may not be a feasible option.

Sign In or Register to comment.