ParameterIterationParallel output

haddockhaddock Member Posts: 849 Maven
edited November 2018 in Help
Greets Folks,

The ParameterIteration operator exposes a value for performance, which can be logged against the values iterated.  As far as I can see, its parallel cousin ParameterIterationParallel does not; moreover, if you log the values it iterates over it seems that all entries reflect only the last values. So if you were iterating one parameter over A, B, and C and another over 1,2,and 3 all process log entries would contain C and 3. Is there a way round this?

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi,
    we are aware of this problem, but its a quite conceptual problem. To solve it, we have to reverse the notification direction: Instead of letting the ProcessLog ask the operator for the value, the operator has to notify the ProcessLog, since there are as many (duplicated) operators as threads, not known by the ProcessLog. But this would be an exception of the typical RapidMiner data flow if the ProcessLogOperator is executed after the value defining operator.
    We are going to solve this issue with RapidMiner 5.0, because some changes on your processes might be needed and we try to aviod this at least during minior version changes.


    Greetings,
      Sebastian
  • haddockhaddock Member Posts: 849 Maven
    Hi Sebastian,

    What an excellent reply, clear and to the point, many thanks. Pondering the question I came to the conclusion that you would need some server style process waiting for output from all those crazy threads, and that I might as well use the SQL data server. There used to be a problem on 4.3 with this, but I expect it has been fixed in 4.3001.

    What I did notice with parallel Parameter iteration was that it used more of the CPU than parallel Evolutionary Parameter Optimisation; now that I'm on double quads I want to use that grunt fully, so I'll iterate in parallel over embedded processes and enjoy the smell of burning silicon.

    Thanks again, onward through the fog!
Sign In or Register to comment.