"Cannot evaluate performance for current parameter combination"

Legacy UserLegacy User Member Posts: 0 Newbie
edited May 2019 in Help
Hi,

I'm doing some GridParameterOptimization for RandomForests. My model
is pretty much the same as the one given in the samples.

Three parameters are evaluated:

RandomForest.number_of_trees, min=1.0, max=100.0, steps=10, logarithmic
RandomForest.criterion, all possible criterion
and RandomForest.maximal_depth, min=-1.0, max=100.0, steps=10, logarithmic.

Sometimes, the learner terminates with:

[NOTE] OperatorChain: Process stopped.
[Warning] GridParameterOptimization: Cannot evaluate performance for current
parameter combination: Process stopped in OperatorChain
[NOTE] GridParameterOptimization: Process stopped.

Why Is it possible that some parameter combinations are not valid? And
how can I figure out which combination failed?
To avoid such "crashes" in the future, is it possible to tell RapidMiner to
ignore these invalid combinations without terminating?

Thanks,
ben
Tagged:

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi Ben,
    could you post your process file here?  Thats probably not the behavior intended :) And at the first glance I don't see, why this can happen...

    Greetings,
      Sebastian
  • Legacy UserLegacy User Member Posts: 0 Newbie
    Hi Sebastian,

    that's my model:

    <operator name="Root" class="Process" expanded="yes">
        <operator name="CSVExampleSource" class="CSVExampleSource">
            <parameter key="filename" value="examples.csv"/>
            <parameter key="label_name" value="mylabel"/>
        </operator>
        <operator name="GridParameterOptimization" class="GridParameterOptimization" expanded="yes">
            <list key="parameters">
              <parameter key="RandomForest.number_of_trees" value="[1.0;100.0;10;logarithmic]"/>
              <parameter key="RandomForest.criterion" value="gain_ratio,information_gain,gini_index,accuracy"/>
              <parameter key="RandomForest.maximal_depth" value="[-1.0;100.0;10;logarithmic]"/>
            </list>
            <operator name="XValidation" class="XValidation" expanded="yes">
                <parameter key="leave_one_out" value="true"/>
                <parameter key="sampling_type" value="shuffled sampling"/>
                <operator name="RandomForest" class="RandomForest">
                    <parameter key="criterion" value="gain_ratio"/>
                    <parameter key="maximal_depth" value="-1"/>
                    <parameter key="number_of_trees" value="5405"/>
                </operator>
                <operator name="OperatorChain" class="OperatorChain" expanded="yes">
                    <operator name="ModelApplier" class="ModelApplier">
                        <list key="application_parameters">
                        </list>
                    </operator>
                    <operator name="ClassificationPerformance" class="ClassificationPerformance">
                        <parameter key="absolute_error" value="true"/>
                        <parameter key="accuracy" value="true"/>
                        <list key="class_weights">
                        </list>
                        <parameter key="classification_error" value="true"/>
                    </operator>
                </operator>
            </operator>
            <operator name="ProcessLog" class="ProcessLog">
                <parameter key="filename" value="/tmp/process.log"/>
                <list key="log">
                  <parameter key="iteration" value="operator.XValidation.value.iteration"/>
                  <parameter key="time" value="operator.XValidation.value.time"/>
                  <parameter key="deviation" value="operator.XValidation.value.deviation"/>
                  <parameter key="variance" value="operator.XValidation.value.variance"/>
                  <parameter key="accuracy" value="operator.XValidation.value.performance"/>
                  <parameter key="classification_error" value="operator.XValidation.value.performance2"/>
                  <parameter key="absolute_error" value="operator.XValidation.value.performance3"/>
                  <parameter key="num_trees" value="operator.RandomForest.parameter.number_of_trees"/>
                  <parameter key="criterion" value="operator.RandomForest.parameter.criterion"/>
                  <parameter key="max_depth" value="operator.RandomForest.parameter.maximal_depth"/>
                </list>
                <parameter key="persistent" value="true"/>
            </operator>
        </operator>
        <operator name="PerformanceWriter" class="PerformanceWriter">
            <parameter key="performance_file" value="/tmp/performance.per"/>
        </operator>
        <operator name="ParameterSetWriter" class="ParameterSetWriter">
            <parameter key="parameter_file" value="/tmp/parameters.par"/>
        </operator>
    </operator>
    Maybe you can find something.

    Thank you.

    Ben
  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi Ben,
    it seems to me, that RapidMiner does not have the permission to write into the file you specified. Did you check that?
    I'm only able to reproduce the error then.

    Greetings,
      Sebastian
Sign In or Register to comment.