"GridParameterOptimization and performances"

UsernameUsername Member Posts: 39 Guru
edited May 2019 in Help
Hi,

how can I use the GridParameterOptimizationOperator to save the PerformanceVectors after each iteration? The result should be the currently selected parameters and the resulting performances, e.g.:

k  Sim  | Accuracy
1  cos    0.3
2  cos    0.5
3  cos    0.4
1  Euc    0.2
2  Euc    0.4
3  Euc    0.3

P.S.: Didn't older RM versions have more similarity measures for the kNN learner?


Thanks
Tagged:

Answers

  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder
    Hello,

    you can use the ProcessLog operator for this (please refer to the sample process "...Meta/01_ParameterOptimization.xml"). Here is the basic setup:

    <operator name="Root" class="Process" expanded="yes">
        <operator name="ExampleSetGenerator" class="ExampleSetGenerator">
            <parameter key="number_examples" value="200"/>
            <parameter key="number_of_attributes" value="2"/>
            <parameter key="target_function" value="checkerboard classification"/>
        </operator>
        <operator name="ParameterOptimization" class="GridParameterOptimization" expanded="yes">
            <list key="parameters">
              <parameter key="NearestNeighbors.k" value="[1.0;3.0;2;linear]"/>
            </list>
            <operator name="Validation" class="XValidation" expanded="yes">
                <parameter key="sampling_type" value="shuffled sampling"/>
                <operator name="NearestNeighbors" class="NearestNeighbors">
                    <parameter key="k" value="3"/>
                </operator>
                <operator name="ApplierChain" class="OperatorChain" expanded="yes">
                    <operator name="Test" class="ModelApplier">
                        <list key="application_parameters">
                        </list>
                    </operator>
                    <operator name="ClassificationPerformance" class="ClassificationPerformance">
                        <parameter key="accuracy" value="true"/>
                        <list key="class_weights">
                        </list>
                    </operator>
                </operator>
            </operator>
            <operator name="Log" class="ProcessLog">
                <parameter key="filename" value="paraopt.log"/>
                <list key="log">
                  <parameter key="k" value="operator.NearestNeighbors.parameter.k"/>
                  <parameter key="Accuracy" value="operator.Validation.value.performance"/>
                </list>
            </operator>
        </operator>
    </operator>
    You can add a filename for the ProcessLog operator into which the results should be written. The results are of course also available in the Log tab of the Results View.

    P.S.: Didn't older RM versions have more similarity measures for the kNN learner?
    Yes, and they will again be integrated. We just completely revised the KNN learner which led to a speed-up of factor 10. However, we had to remove the usual similarity measure in order to get this speed-up and did not manage to re-implement all measures for KNN yet. But (at least most of them) will be available again in future releases.

    Cheers,
    Ingo
  • UsernameUsername Member Posts: 39 Guru
    Thanks for your answer. I hope the cosine similarity will be back soon.
Sign In or Register to comment.