How can I insert a Lift Chart into a Validation

marcopomarcopo Member Posts: 14 Contributor II
edited November 2018 in Help
Hello Community,
I saw this both interesting videos about ROC and LIFT charts. In this videos the LIFT charts appears all the time after a apply model.



Nevertheless, in both videos they are not using a validation. I am trying to use the validation and a Lift Chart but I am getting all the time errors? I tried to insert the lift chart after the apply model in the validation but in the end I get this errors. What can I do?

Expected AverageVector but received ExampleSet.
Expected AverageVector but received DistributionModel.
Expected AverageVector but received LiftParetoChart.

In addition I am looking for a way or setting, that in the results example set an additionally row is inserted which shows the predicted class. So I can see which value is true and which one is false.

Best Regards and thank you



  • Options
    marcopomarcopo Member Posts: 14 Contributor II
    No one an idea?
  • Options
    MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn

    as always, it would be a good idea to attach the process you have so far to your post as explained in my signature.

    Concerning the Lift Chart, what do you expect from a lift chart in a validation? What do you want to achieve?

    Best regards,
  • Options
    marcopomarcopo Member Posts: 14 Contributor II

    actually I wanna see the original class and the class which is predicted through the model. Additionally, I wanna use the lift-chart as performance indicator.

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <process version="5.2.008">
      <operator activated="true" class="process" compatibility="5.2.008" expanded="true" name="Process">
        <process expanded="true" height="755" width="1015">
          <operator activated="true" class="retrieve" compatibility="5.2.008" expanded="true" height="60" name="Retrieve" width="90" x="45" y="390">
            <parameter key="repository_entry" value="../../Daten/TextMiningAusgang0MinBinominaNeu"/>
          <operator activated="true" class="split_validation" compatibility="5.2.008" expanded="true" height="148" name="Validation (2)" width="90" x="410" y="293">
            <parameter key="parallelize_training" value="true"/>
            <parameter key="parallelize_testing" value="true"/>
            <process expanded="true" height="759" width="483">
              <operator activated="true" class="nominal_to_numerical" compatibility="5.2.008" expanded="true" height="94" name="Nominal to Numerical (2)" width="90" x="45" y="30">
                <list key="comparison_groups"/>
              <operator activated="true" class="support_vector_machine_libsvm" compatibility="5.2.008" expanded="true" height="76" name="SVM" width="90" x="264" y="30">
                <parameter key="svm_type" value="nu-SVC"/>
                <parameter key="C" value="2.0"/>
                <list key="class_weights"/>
              <connect from_port="training" to_op="Nominal to Numerical (2)" to_port="example set input"/>
              <connect from_op="Nominal to Numerical (2)" from_port="example set output" to_op="SVM" to_port="training set"/>
              <connect from_op="SVM" from_port="model" to_port="model"/>
              <portSpacing port="source_training" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_through 1" spacing="0"/>
            <process expanded="true" height="759" width="483">
              <operator activated="true" class="apply_model" compatibility="5.2.008" expanded="true" height="76" name="Apply Model (2)" width="90" x="45" y="30">
                <list key="application_parameters"/>
              <operator activated="true" class="create_lift_chart" compatibility="5.2.008" expanded="true" height="94" name="Create Lift Chart" width="90" x="112" y="390">
                <parameter key="target_class" value="UP"/>
              <operator activated="true" class="performance_binominal_classification" compatibility="5.2.008" expanded="true" height="76" name="Performance (2)" width="90" x="246" y="30">
                <parameter key="classification_error" value="true"/>
                <parameter key="precision" value="true"/>
                <parameter key="recall" value="true"/>
                <parameter key="f_measure" value="true"/>
                <parameter key="sensitivity" value="true"/>
                <parameter key="specificity" value="true"/>
              <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
              <connect from_port="test set" to_op="Apply Model (2)" to_port="unlabelled data"/>
              <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
              <connect from_op="Apply Model (2)" from_port="model" to_op="Create Lift Chart" to_port="model"/>
              <connect from_op="Create Lift Chart" from_port="lift pareto chart" to_port="averagable 2"/>
              <connect from_op="Performance (2)" from_port="performance" to_port="averagable 1"/>
              <connect from_op="Performance (2)" from_port="example set" to_port="averagable 3"/>
              <portSpacing port="source_model" spacing="0"/>
              <portSpacing port="source_test set" spacing="0"/>
              <portSpacing port="source_through 1" spacing="0"/>
              <portSpacing port="sink_averagable 1" spacing="0"/>
              <portSpacing port="sink_averagable 2" spacing="0"/>
              <portSpacing port="sink_averagable 3" spacing="0"/>
              <portSpacing port="sink_averagable 4" spacing="0"/>
          <connect from_op="Retrieve" from_port="output" to_op="Validation (2)" to_port="training"/>
          <connect from_op="Validation (2)" from_port="averagable 1" to_port="result 1"/>
          <connect from_op="Validation (2)" from_port="averagable 2" to_port="result 2"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
          <portSpacing port="sink_result 3" spacing="0"/>
    Best Regards

  • Options
    MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn

    you can't use the Lift Chart as output of the X-Validation. As you know, the cross validation executes it's inner processes 10 times and then builds the average of the performances of all 10 iterations. But averaging a Lift Chart is not possible.
    You would have to create a model outside of the validation and then create the Lift Chart based on that model.

    I hope this helps!

    Best regards,
Sign In or Register to comment.