"Precision/Recall charts"

svm_friendsvm_friend Member Posts: 3 Contributor I
edited May 2019 in Help
Is it possible to create 2D-Precision/Recall charts like this on:

http://proteomics.bioengr.uic.edu/malibu/docs/images/evaluate/spr_thumb.png

All I found were ROC-curves

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi,
    I'm not quite sure, that this is exactly what you want. But you could log the precision / recall and plot it using the plotter as you plot your data.
    Here's a process showing how to log them:
    <operator name="Root" class="Process" expanded="yes">
        <description text="#ylt#p#ygt# Often the different operators have many parameters and it is not clear which parameter values are best for the learning task at hand. The parameter optimization operator helps to find an optimal parameter set for the used operators. #ylt#/p#ygt#  #ylt#p#ygt# The inner crossvalidation estimates the performance for each parameter set. In this experiment two parameters of the SVM are tuned. The result can be plotted in 3D (using gnuplot) or in color mode. #ylt#/p#ygt#  #ylt#p#ygt# Try the following: #ylt#ul#ygt# #ylt#li#ygt#Start the experiment. The result is the best parameter set and the performance which was achieved with this parameter set.#ylt#/li#ygt# #ylt#li#ygt#Edit the parameter list of the ParameterOptimization operator to find another parameter set.#ylt#/li#ygt# #ylt#/ul#ygt# #ylt#/p#ygt# "/>
        <operator name="Input" class="ExampleSource">
            <parameter key="attributes" value="..\data\iris.aml"/>
        </operator>
        <operator name="ParameterOptimization" class="GridParameterOptimization" expanded="yes">
            <list key="parameters">
              <parameter key="Training.C" value="50,100,150,200,250"/>
              <parameter key="Training.degree" value="1,2,3,4,5"/>
            </list>
            <operator name="Validation" class="XValidation" expanded="yes">
                <parameter key="sampling_type" value="shuffled sampling"/>
                <operator name="Training" class="LibSVMLearner">
                    <parameter key="C" value="250.0"/>
                    <list key="class_weights">
                    </list>
                    <parameter key="degree" value="5"/>
                    <parameter key="epsilon" value="0.01"/>
                    <parameter key="kernel_type" value="poly"/>
                </operator>
                <operator name="ApplierChain" class="OperatorChain" expanded="yes">
                    <operator name="Test" class="ModelApplier">
                        <list key="application_parameters">
                        </list>
                    </operator>
                    <operator name="ClassificationPerformance" class="ClassificationPerformance">
                        <list key="class_weights">
                        </list>
                        <parameter key="main_criterion" value="weighted_mean_recall"/>
                        <parameter key="weighted_mean_precision" value="true"/>
                        <parameter key="weighted_mean_recall" value="true"/>
                    </operator>
                </operator>
            </operator>
            <operator name="Log" class="ProcessLog">
                <parameter key="filename" value="paraopt.log"/>
                <list key="log">
                  <parameter key="C" value="operator.Training.parameter.C"/>
                  <parameter key="degree" value="operator.Training.parameter.degree"/>
                  <parameter key="recall" value="operator.Validation.value.performance1"/>
                  <parameter key="precision" value="operator.Validation.value.performance2"/>
                </list>
            </operator>
        </operator>
    </operator>
    Greetings,
      Sebastian
  • emaema Member Posts: 33 Maven
    In case of other classifiers like nearest neighbors and others
    what parameters should we consider
  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi Ema,
    do you ask, which parameters you should log into the processLog? I think this depends on your task at hand. Values you never change probably don't give you much information. So include everything you might optimize or change during the process.

    Greetings,
      Sebastian
Sign In or Register to comment.