image

🎉 🎉 RAPIDMINER 9.10 IS OUT!!! 🎉🎉

Download the latest version helping analytics teams accelerate time-to-value for streaming and IIOT use cases.

CLICK HERE TO DOWNLOAD

"How do you apply Platt Scaling in X-Validation?"

Charles54Charles54 Member Posts: 8 Contributor II
edited May 2019 in Help
Hello all,

I am having trouble using the Rescale Confidences operator. I looked over the sample file, and it seems straight forward,, but I can't figure out how to apply it in an X-Validation.

This is the best I can come up with. You can see that the labeled data which is output from the Apply Model operator does not contain confidence values -- therefore the Performance operator fails. I am new to data mining, so perhaps my thinking is way off the mark. (I used Rapid Miner 5)
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
 <context>
   <input/>
   <output/>
   <macros/>
 </context>
 <operator activated="true" class="process" expanded="true" name="Root">
   <process expanded="true" height="395" width="748">
     <operator activated="true" class="generate_data" expanded="true" height="60" name="ExampleSetGenerator" width="90" x="45" y="30">
       <parameter key="target_function" value="checkerboard classification"/>
       <parameter key="number_examples" value="500"/>
       <parameter key="number_of_attributes" value="2"/>
     </operator>
     <operator activated="true" class="x_validation" expanded="true" height="112" name="Validation" width="90" x="246" y="30">
       <process expanded="true" height="414" width="433">
         <operator activated="true" class="support_vector_machine" expanded="true" height="112" name="JMySVMLearner" width="90" x="45" y="30">
           <parameter key="kernel_type" value="radial"/>
         </operator>
         <operator activated="true" class="rescale_confidences" expanded="true" height="76" name="PlattScaling" width="90" x="179" y="30"/>
         <connect from_port="training" to_op="JMySVMLearner" to_port="training set"/>
         <connect from_op="JMySVMLearner" from_port="model" to_op="PlattScaling" to_port="prediction model"/>
         <connect from_op="JMySVMLearner" from_port="exampleSet" to_op="PlattScaling" to_port="example set"/>
         <connect from_op="PlattScaling" from_port="model" to_port="model"/>
         <portSpacing port="source_training" spacing="0"/>
         <portSpacing port="sink_model" spacing="0"/>
         <portSpacing port="sink_through 1" spacing="0"/>
       </process>
       <process expanded="true" height="414" width="435">
         <operator activated="true" class="apply_model" expanded="true" height="76" name="Apply Model" width="90" x="45" y="30">
           <list key="application_parameters"/>
         </operator>
         <operator activated="true" class="rescale_confidences" expanded="true" height="76" name="Rescale Confidences" width="90" x="179" y="30"/>
         <operator activated="true" class="performance" expanded="true" height="76" name="Performance" width="90" x="313" y="30"/>
         <connect from_port="model" to_op="Apply Model" to_port="model"/>
         <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
         <connect from_op="Apply Model" from_port="labelled data" to_op="Rescale Confidences" to_port="example set"/>
         <connect from_op="Apply Model" from_port="model" to_op="Rescale Confidences" to_port="prediction model"/>
         <connect from_op="Rescale Confidences" from_port="example set" to_op="Performance" to_port="labelled data"/>
         <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
         <portSpacing port="source_model" spacing="0"/>
         <portSpacing port="source_test set" spacing="0"/>
         <portSpacing port="source_through 1" spacing="0"/>
         <portSpacing port="sink_averagable 1" spacing="0"/>
         <portSpacing port="sink_averagable 2" spacing="0"/>
         <portSpacing port="sink_averagable 3" spacing="0"/>
       </process>
     </operator>
     <connect from_op="ExampleSetGenerator" from_port="output" to_op="Validation" to_port="training"/>
     <connect from_op="Validation" from_port="model" to_port="result 1"/>
     <connect from_op="Validation" from_port="averagable 1" to_port="result 2"/>
     <portSpacing port="source_input 1" spacing="0"/>
     <portSpacing port="sink_result 1" spacing="0"/>
     <portSpacing port="sink_result 2" spacing="0"/>
     <portSpacing port="sink_result 3" spacing="0"/>
   </process>
 </operator>
</process>

I have read over Steffen's posts on this subject, but I am afraid I still can't figure it out. Any help would be much appreciated.

Regards, Charles

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531   Unicorn
    Hi Charles,
    just delete the PlatScaling. The second one will do the trick. PlatScaling internally applies the model, but does not alter it. Here's how it works:
    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <process version="5.0">
      <context>
        <input>
          <location/>
        </input>
        <output>
          <location/>
          <location/>
          <location/>
        </output>
        <macros/>
      </context>
      <operator activated="true" class="process" expanded="true" name="Root">
        <process expanded="true" height="395" width="748">
          <operator activated="true" class="generate_data" expanded="true" height="60" name="ExampleSetGenerator" width="90" x="45" y="30">
            <parameter key="target_function" value="checkerboard classification"/>
            <parameter key="number_examples" value="500"/>
            <parameter key="number_of_attributes" value="2"/>
          </operator>
          <operator activated="true" class="x_validation" expanded="true" height="112" name="Validation" width="90" x="246" y="30">
            <process expanded="true" height="414" width="533">
              <operator activated="true" class="support_vector_machine" expanded="true" height="112" name="JMySVMLearner" width="90" x="45" y="30">
                <parameter key="kernel_type" value="radial"/>
              </operator>
              <connect from_port="training" to_op="JMySVMLearner" to_port="training set"/>
              <connect from_op="JMySVMLearner" from_port="model" to_port="model"/>
              <portSpacing port="source_training" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_through 1" spacing="0"/>
            </process>
            <process expanded="true" height="414" width="435">
              <operator activated="true" class="rescale_confidences" expanded="true" height="76" name="PlattScaling" width="90" x="45" y="30"/>
              <operator activated="true" class="performance" expanded="true" height="76" name="Performance" width="90" x="313" y="30"/>
              <connect from_port="model" to_op="PlattScaling" to_port="prediction model"/>
              <connect from_port="test set" to_op="PlattScaling" to_port="example set"/>
              <connect from_op="PlattScaling" from_port="example set" to_op="Performance" to_port="labelled data"/>
              <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
              <portSpacing port="source_model" spacing="0"/>
              <portSpacing port="source_test set" spacing="0"/>
              <portSpacing port="source_through 1" spacing="0"/>
              <portSpacing port="sink_averagable 1" spacing="0"/>
              <portSpacing port="sink_averagable 2" spacing="0"/>
            </process>
          </operator>
          <connect from_op="ExampleSetGenerator" from_port="output" to_op="Validation" to_port="training"/>
          <connect from_op="Validation" from_port="model" to_port="result 1"/>
          <connect from_op="Validation" from_port="averagable 1" to_port="result 2"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
          <portSpacing port="sink_result 3" spacing="0"/>
        </process>
      </operator>
    </process>
    Greetings,
      Sebastian
  • Charles54Charles54 Member Posts: 8 Contributor II
    Hello Sebastian,

    Thanks so much for the clear - and very quick - reply. I had not realized that the Platt scaling operator replaced the need for the model applier. As usual, I understand the complex eventually... the obvious takes me a little longer.

    Unfortunately, the configuration your offered gives me the same wonky output as my original process. However, at least I know that the problem lies somewhere other than with the scaling. I will try running it again with a larger sample. Thanks again for sharing your expertise. It probably saved me numerous hours of futile experimentation. Have a great day.

    Regards, Charles
Sign In or Register to comment.