RapidMiner

RapidMiner

Cannot evaluate performance for current parameter combination

Cannot evaluate performance for current parameter combination

Hi,

I'm doing some GridParameterOptimization for RandomForests. My model
is pretty much the same as the one given in the samples.

Three parameters are evaluated:

RandomForest.number_of_trees, min=1.0, max=100.0, steps=10, logarithmic
RandomForest.criterion, all possible criterion
and RandomForest.maximal_depth, min=-1.0, max=100.0, steps=10, logarithmic.

Sometimes, the learner terminates with:

[NOTE] OperatorChain: Process stopped.
[Warning] GridParameterOptimization: Cannot evaluate performance for current
parameter combination: Process stopped in OperatorChain
[NOTE] GridParameterOptimization: Process stopped.

Why Is it possible that some parameter combinations are not valid? And
how can I figure out which combination failed?
To avoid such "crashes" in the future, is it possible to tell RapidMiner to
ignore these invalid combinations without terminating?

Thanks,
ben
3 REPLIES
Elite

Re: Cannot evaluate performance for current parameter combination

Hi Ben,
could you post your process file here?  Thats probably not the behavior intended Smiley Happy And at the first glance I don't see, why this can happen...

Greetings,
  Sebastian
Old World Computing - Establishing the Future

Check out the Jackhammer Extension for RapidMiner! Crunch more data easier and with up to 700% speed up! Available only here

Re: Cannot evaluate performance for current parameter combination

Hi Sebastian,

that's my model:


<operator name="Root" class="Process" expanded="yes">
    <operator name="CSVExampleSource" class="CSVExampleSource">
        <parameter key="filename" value="examples.csv"/>
        <parameter key="label_name" value="mylabel"/>
    </operator>
    <operator name="GridParameterOptimization" class="GridParameterOptimization" expanded="yes">
        <list key="parameters">
          <parameter key="RandomForest.number_of_trees" value="[1.0;100.0;10;logarithmic]"/>
          <parameter key="RandomForest.criterion" value="gain_ratio,information_gain,gini_index,accuracy"/>
          <parameter key="RandomForest.maximal_depth" value="[-1.0;100.0;10;logarithmic]"/>
        </list>
        <operator name="XValidation" class="XValidation" expanded="yes">
            <parameter key="leave_one_out" value="true"/>
            <parameter key="sampling_type" value="shuffled sampling"/>
            <operator name="RandomForest" class="RandomForest">
                <parameter key="criterion" value="gain_ratio"/>
                <parameter key="maximal_depth" value="-1"/>
                <parameter key="number_of_trees" value="5405"/>
            </operator>
            <operator name="OperatorChain" class="OperatorChain" expanded="yes">
                <operator name="ModelApplier" class="ModelApplier">
                    <list key="application_parameters">
                    </list>
                </operator>
                <operator name="ClassificationPerformance" class="ClassificationPerformance">
                    <parameter key="absolute_error" value="true"/>
                    <parameter key="accuracy" value="true"/>
                    <list key="class_weights">
                    </list>
                    <parameter key="classification_error" value="true"/>
                </operator>
            </operator>
        </operator>
        <operator name="ProcessLog" class="ProcessLog">
            <parameter key="filename" value="/tmp/process.log"/>
            <list key="log">
              <parameter key="iteration" value="operator.XValidation.value.iteration"/>
              <parameter key="time" value="operator.XValidation.value.time"/>
              <parameter key="deviation" value="operator.XValidation.value.deviation"/>
              <parameter key="variance" value="operator.XValidation.value.variance"/>
              <parameter key="accuracy" value="operator.XValidation.value.performance"/>
              <parameter key="classification_error" value="operator.XValidation.value.performance2"/>
              <parameter key="absolute_error" value="operator.XValidation.value.performance3"/>
              <parameter key="num_trees" value="operator.RandomForest.parameter.number_of_trees"/>
              <parameter key="criterion" value="operator.RandomForest.parameter.criterion"/>
              <parameter key="max_depth" value="operator.RandomForest.parameter.maximal_depth"/>
            </list>
            <parameter key="persistent" value="true"/>
        </operator>
    </operator>
    <operator name="PerformanceWriter" class="PerformanceWriter">
        <parameter key="performance_file" value="/tmp/performance.per"/>
    </operator>
    <operator name="ParameterSetWriter" class="ParameterSetWriter">
        <parameter key="parameter_file" value="/tmp/parameters.par"/>
    </operator>
</operator>


Maybe you can find something.

Thank you.

Ben
Elite

Re: Cannot evaluate performance for current parameter combination

Hi Ben,
it seems to me, that RapidMiner does not have the permission to write into the file you specified. Did you check that?
I'm only able to reproduce the error then.

Greetings,
  Sebastian
Old World Computing - Establishing the Future

Check out the Jackhammer Extension for RapidMiner! Crunch more data easier and with up to 700% speed up! Available only here