decision tree does not fit the roc

cam8cam8 Member Posts: 2 Contributor I
edited November 2018 in Help

Hello,

 

i work actually at a data set for a decision tree solution inspired from your video "finding the right model".

 

The roc based on the data set shows me a good result for decision tree and random forest.

But when i want to create a simple decision tree model i get a very low recall of 1,12% for TruePositiv.

With Optimize Parameters i can increase it to 47% for TruePositiv.

But i see in documentations, that a TruePositiv of 99% is possible.

 

Is something wrong with my process?

 

The XML-Code:

<?xml version="1.0" encoding="UTF-8"?><process version="8.2.000">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="8.2.000" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="retrieve" compatibility="8.2.000" expanded="true" height="68" name="Retrieve transfusion.dataset1" width="90" x="45" y="34">
<parameter key="repository_entry" value="../data/transfusion.dataset1"/>
</operator>
<operator activated="true" class="rename" compatibility="8.2.000" expanded="true" height="82" name="Rename" width="90" x="179" y="34">
<parameter key="old_name" value="whether he/she donated blood in March 2007"/>
<parameter key="new_name" value="Donation"/>
<list key="rename_additional_attributes">
<parameter key="Frequency (times)" value="Frequency"/>
<parameter key="Monetary (c.c. blood)" value="Monetary"/>
<parameter key="Recency (months)" value="Recency"/>
<parameter key="Time (months)" value="Time"/>
<parameter key="ID (customer)" value="ID"/>
</list>
</operator>
<operator activated="true" class="numerical_to_binominal" compatibility="8.2.000" expanded="true" height="82" name="Numerical to Binominal" width="90" x="313" y="34">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="Donation"/>
</operator>
<operator activated="true" class="set_role" compatibility="8.2.000" expanded="true" height="82" name="Set Role" width="90" x="447" y="34">
<parameter key="attribute_name" value="Donation"/>
<parameter key="target_role" value="label"/>
<list key="set_additional_roles">
<parameter key="ID" value="id"/>
<parameter key="Frequency" value="regular"/>
<parameter key="Monetary" value="regular"/>
<parameter key="Recency" value="regular"/>
<parameter key="Time" value="regular"/>
</list>
</operator>
<operator activated="true" class="select_attributes" compatibility="8.2.000" expanded="true" height="82" name="Select Attributes" width="90" x="581" y="34">
<parameter key="attribute_filter_type" value="subset"/>
<parameter key="attributes" value="Donation|Frequency|Recency|Time|Monetary"/>
<parameter key="include_special_attributes" value="true"/>
</operator>
<operator activated="true" class="multiply" compatibility="8.2.000" expanded="true" height="124" name="Multiply (2)" width="90" x="715" y="34"/>
<operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="8.2.000" expanded="true" height="124" name="DT Optimize Parameters" width="90" x="849" y="289">
<list key="parameters">
<parameter key="Decision Tree (2).criterion" value="gini_index"/>
<parameter key="Decision Tree (2).maximal_depth" value="[15;30;15;linear]"/>
<parameter key="Decision Tree (2).minimal_gain" value="[0.040;0.050;10;linear]"/>
<parameter key="Decision Tree (2).apply_prepruning" value="true"/>
<parameter key="Decision Tree (2).minimal_size_for_split" value="[5;10;6;linear]"/>
<parameter key="Decision Tree (2).number_of_prepruning_alternatives" value="[10;15;5;linear]"/>
</list>
<parameter key="log_all_criteria" value="true"/>
<process expanded="true">
<operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="DT Cross Validation (2)" width="90" x="45" y="34">
<process expanded="true">
<operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (2)" width="90" x="112" y="34">
<parameter key="criterion" value="gini_index"/>
</operator>
<connect from_port="training set" to_op="Decision Tree (2)" to_port="training set"/>
<connect from_op="Decision Tree (2)" from_port="model" to_port="model"/>
<portSpacing port="source_training set" spacing="0"/>
<portSpacing port="sink_model" spacing="0"/>
<portSpacing port="sink_through 1" spacing="0"/>
</process>
<process expanded="true">
<operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model (4)" width="90" x="112" y="34">
<list key="application_parameters"/>
</operator>
<operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance (4)" width="90" x="246" y="34">
<parameter key="main_criterion" value="recall"/>
<parameter key="AUC (optimistic)" value="true"/>
<parameter key="AUC" value="true"/>
<parameter key="AUC (pessimistic)" value="true"/>
<parameter key="precision" value="true"/>
<parameter key="recall" value="true"/>
</operator>
<connect from_port="model" to_op="Apply Model (4)" to_port="model"/>
<connect from_port="test set" to_op="Apply Model (4)" to_port="unlabelled data"/>
<connect from_op="Apply Model (4)" from_port="labelled data" to_op="Performance (4)" to_port="labelled data"/>
<connect from_op="Performance (4)" from_port="performance" to_port="performance 1"/>
<portSpacing port="source_model" spacing="0"/>
<portSpacing port="source_test set" spacing="0"/>
<portSpacing port="source_through 1" spacing="0"/>
<portSpacing port="sink_test set results" spacing="0"/>
<portSpacing port="sink_performance 1" spacing="0"/>
<portSpacing port="sink_performance 2" spacing="0"/>
</process>
</operator>
<connect from_port="input 1" to_op="DT Cross Validation (2)" to_port="example set"/>
<connect from_op="DT Cross Validation (2)" from_port="model" to_port="model"/>
<connect from_op="DT Cross Validation (2)" from_port="performance 1" to_port="performance"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_performance" spacing="0"/>
<portSpacing port="sink_model" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
</process>
</operator>
<operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="Cross Validation" width="90" x="849" y="136">
<process expanded="true">
<operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree" width="90" x="112" y="34"/>
<connect from_port="training set" to_op="Decision Tree" to_port="training set"/>
<connect from_op="Decision Tree" from_port="model" to_port="model"/>
<portSpacing port="source_training set" spacing="0"/>
<portSpacing port="sink_model" spacing="0"/>
<portSpacing port="sink_through 1" spacing="0"/>
</process>
<process expanded="true">
<operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model" width="90" x="112" y="34">
<list key="application_parameters"/>
</operator>
<operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance" width="90" x="246" y="34">
<parameter key="main_criterion" value="recall"/>
<parameter key="precision" value="true"/>
<parameter key="recall" value="true"/>
</operator>
<connect from_port="model" to_op="Apply Model" to_port="model"/>
<connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
<connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
<connect from_op="Performance" from_port="performance" to_port="performance 1"/>
<portSpacing port="source_model" spacing="0"/>
<portSpacing port="source_test set" spacing="0"/>
<portSpacing port="source_through 1" spacing="0"/>
<portSpacing port="sink_test set results" spacing="0"/>
<portSpacing port="sink_performance 1" spacing="0"/>
<portSpacing port="sink_performance 2" spacing="0"/>
</process>
</operator>
<operator activated="true" class="compare_rocs" compatibility="8.2.000" expanded="true" height="82" name="Compare ROCs (3)" width="90" x="849" y="34">
<parameter key="sampling_type" value="automatic"/>
<parameter key="use_example_weights" value="false"/>
<process expanded="true">
<operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (5)" width="90" x="447" y="34"/>
<operator activated="true" class="naive_bayes" compatibility="8.2.000" expanded="true" height="82" name="Naive Bayes (3)" width="90" x="447" y="136"/>
<operator activated="true" class="rule_induction" compatibility="8.2.000" expanded="true" height="82" name="Rule Induction (3)" width="90" x="447" y="238"/>
<operator activated="true" class="random_tree" compatibility="8.2.000" expanded="true" height="82" name="Random Tree (3)" width="90" x="447" y="340"/>
<operator activated="true" class="k_nn" compatibility="8.2.000" expanded="true" height="82" name="k-NN (5)" width="90" x="447" y="442"/>
<connect from_port="train 1" to_op="Decision Tree (5)" to_port="training set"/>
<connect from_port="train 2" to_op="Naive Bayes (3)" to_port="training set"/>
<connect from_port="train 3" to_op="Rule Induction (3)" to_port="training set"/>
<connect from_port="train 4" to_op="Random Tree (3)" to_port="training set"/>
<connect from_port="train 5" to_op="k-NN (5)" to_port="training set"/>
<connect from_op="Decision Tree (5)" from_port="model" to_port="model 1"/>
<connect from_op="Naive Bayes (3)" from_port="model" to_port="model 2"/>
<connect from_op="Rule Induction (3)" from_port="model" to_port="model 3"/>
<connect from_op="Random Tree (3)" from_port="model" to_port="model 4"/>
<connect from_op="k-NN (5)" from_port="model" to_port="model 5"/>
<portSpacing port="source_train 1" spacing="0"/>
<portSpacing port="source_train 2" spacing="0"/>
<portSpacing port="source_train 3" spacing="0"/>
<portSpacing port="source_train 4" spacing="0"/>
<portSpacing port="source_train 5" spacing="0"/>
<portSpacing port="source_train 6" spacing="0"/>
<portSpacing port="sink_model 1" spacing="0"/>
<portSpacing port="sink_model 2" spacing="0"/>
<portSpacing port="sink_model 3" spacing="0"/>
<portSpacing port="sink_model 4" spacing="0"/>
<portSpacing port="sink_model 5" spacing="0"/>
<portSpacing port="sink_model 6" spacing="0"/>
</process>
</operator>
<connect from_op="Retrieve transfusion.dataset1" from_port="output" to_op="Rename" to_port="example set input"/>
<connect from_op="Rename" from_port="example set output" to_op="Numerical to Binominal" to_port="example set input"/>
<connect from_op="Numerical to Binominal" from_port="example set output" to_op="Set Role" to_port="example set input"/>
<connect from_op="Set Role" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
<connect from_op="Select Attributes" from_port="example set output" to_op="Multiply (2)" to_port="input"/>
<connect from_op="Multiply (2)" from_port="output 1" to_op="Compare ROCs (3)" to_port="example set"/>
<connect from_op="Multiply (2)" from_port="output 2" to_op="Cross Validation" to_port="example set"/>
<connect from_op="Multiply (2)" from_port="output 3" to_op="DT Optimize Parameters" to_port="input 1"/>
<connect from_op="DT Optimize Parameters" from_port="performance" to_port="result 4"/>
<connect from_op="DT Optimize Parameters" from_port="model" to_port="result 5"/>
<connect from_op="Cross Validation" from_port="model" to_port="result 2"/>
<connect from_op="Cross Validation" from_port="performance 1" to_port="result 3"/>
<connect from_op="Compare ROCs (3)" from_port="rocComparison" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
<portSpacing port="sink_result 4" spacing="0"/>
<portSpacing port="sink_result 5" spacing="0"/>
<portSpacing port="sink_result 6" spacing="0"/>
</process>
</operator>
</process>


roc.png

Tagged:

Best Answer

  • lionelderkrikorlionelderkrikor Posts: 617   Unicorn
    Solution Accepted

    Hi @cam8,

     

    There isn't wrong in your process.

    You have an imbalanced dataset (ratio true/false = 178/570). That's lead generally to a bad Recall.

    To increase the Recall, you can sample your data. By sampling your data, I'm obtaining Recall = 60% with a decision tree.

    However, if your goal is the research of the maximum performance, decision tree is not the best choice : a better choice

    is the kNN model. After training this model, you can obtain a recall > 70 %, (with  1< k < 10) and you can obtain a theoretical recall > 90% by optimizing the value of k. But use with caution, this last case correspond to a situation of overfitting : Your model has, in deed, good performances on your training dataset, but will have bad performances on future 'unseen data'.

     

    I hope it helps,

     

    Regards,

     

    Lionel

     

    NB : The process:

    <?xml version="1.0" encoding="UTF-8"?><process version="8.2.000">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="8.2.000" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="read_csv" compatibility="8.2.000" expanded="true" height="68" name="Read CSV" width="90" x="45" y="34">
    <parameter key="csv_file" value="C:\Users\Lionel\Documents\Formations_DataScience\Rapidminer\Tests_Rapidminer\Donators_DT\Donators.csv"/>
    <parameter key="column_separators" value=",\s*|;\s*"/>
    <parameter key="first_row_as_names" value="false"/>
    <list key="annotations">
    <parameter key="0" value="Name"/>
    </list>
    <parameter key="encoding" value="windows-1252"/>
    <list key="data_set_meta_data_information">
    <parameter key="0" value="R&#0;e&#0;c&#0;e&#0;n&#0;c&#0;y&#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;).true.polynominal.attribute"/>
    <parameter key="1" value="F&#0;r&#0;e&#0;q&#0;u&#0;e&#0;n&#0;c&#0;y&#0; &#0;(&#0;t&#0;i&#0;m&#0;e&#0;s&#0;).true.polynominal.attribute"/>
    <parameter key="2" value="M&#0;o&#0;n&#0;e&#0;t&#0;a&#0;r&#0;y&#0; &#0;(&#0;c&#0;\.&#0;c&#0;\.&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0;).true.polynominal.attribute"/>
    <parameter key="3" value="T&#0;i&#0;m&#0;e&#0; &#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;).true.polynominal.attribute"/>
    <parameter key="4" value="w&#0;h&#0;e&#0;t&#0;h&#0;e&#0;r&#0; &#0;h&#0;e&#0;/&#0;s&#0;h&#0;e&#0; &#0;d&#0;o&#0;n&#0;a&#0;t&#0;e&#0;d&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0; &#0;i&#0;n&#0; &#0;M&#0;a&#0;r&#0;c&#0;h&#0; &#0;2&#0;0&#0;0&#0;7.true.integer.attribute"/>
    </list>
    </operator>
    <operator activated="true" class="rename" compatibility="8.2.000" expanded="true" height="82" name="Rename" width="90" x="179" y="34">
    <parameter key="old_name" value="w&#0;h&#0;e&#0;t&#0;h&#0;e&#0;r&#0; &#0;h&#0;e&#0;/&#0;s&#0;h&#0;e&#0; &#0;d&#0;o&#0;n&#0;a&#0;t&#0;e&#0;d&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0; &#0;i&#0;n&#0; &#0;M&#0;a&#0;r&#0;c&#0;h&#0; &#0;2&#0;0&#0;0&#0;7"/>
    <parameter key="new_name" value="Donation"/>
    <list key="rename_additional_attributes">
    <parameter key="F&#0;r&#0;e&#0;q&#0;u&#0;e&#0;n&#0;c&#0;y&#0; &#0;(&#0;t&#0;i&#0;m&#0;e&#0;s&#0;)" value="Frequency"/>
    <parameter key="M&#0;o&#0;n&#0;e&#0;t&#0;a&#0;r&#0;y&#0; &#0;(&#0;c&#0;.&#0;c&#0;.&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0;)" value="Monetary"/>
    <parameter key="R&#0;e&#0;c&#0;e&#0;n&#0;c&#0;y&#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;)" value="Recency"/>
    <parameter key="T&#0;i&#0;m&#0;e&#0; &#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;)" value="Time"/>
    </list>
    </operator>
    <operator activated="true" class="numerical_to_binominal" compatibility="8.2.000" expanded="true" height="82" name="Numerical to Binominal" width="90" x="313" y="34">
    <parameter key="attribute_filter_type" value="single"/>
    <parameter key="attribute" value="Donation"/>
    </operator>
    <operator activated="true" class="set_role" compatibility="8.2.000" expanded="true" height="82" name="Set Role" width="90" x="447" y="34">
    <parameter key="attribute_name" value="Donation"/>
    <parameter key="target_role" value="label"/>
    <list key="set_additional_roles"/>
    </operator>
    <operator activated="true" class="select_attributes" compatibility="8.2.000" expanded="true" height="82" name="Select Attributes" width="90" x="581" y="34">
    <parameter key="attribute_filter_type" value="subset"/>
    <parameter key="attributes" value="Donation|Frequency|Recency|Time|Monetary"/>
    <parameter key="include_special_attributes" value="true"/>
    </operator>
    <operator activated="true" class="multiply" compatibility="8.2.000" expanded="true" height="187" name="Multiply (2)" width="90" x="313" y="187"/>
    <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="8.2.000" expanded="true" height="124" name="DT Optimize Parameters" width="90" x="849" y="289">
    <list key="parameters">
    <parameter key="Decision Tree (2).criterion" value="gain_ratio"/>
    <parameter key="Sample (2).sample_size" value="[30;40;10;linear]"/>
    <parameter key="Decision Tree (2).confidence" value="[0.45;0.5;10;linear]"/>
    <parameter key="Decision Tree (2).minimal_leaf_size" value="[1.0;5;10;linear]"/>
    <parameter key="Decision Tree (2).minimal_size_for_split" value="[1.0;100.0;10;linear]"/>
    </list>
    <process expanded="true">
    <operator activated="true" class="extract_macro" compatibility="8.2.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="112" y="34">
    <parameter key="macro" value="exampleCount"/>
    <parameter key="macro_type" value="statistics"/>
    <parameter key="statistics" value="count"/>
    <parameter key="attribute_name" value="Donation"/>
    <parameter key="attribute_value" value="true"/>
    <list key="additional_macros"/>
    </operator>
    <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="DT Cross Validation (2)" width="90" x="581" y="34">
    <process expanded="true">
    <operator activated="true" class="sample" compatibility="8.2.000" expanded="true" height="82" name="Sample (2)" width="90" x="179" y="34">
    <parameter key="balance_data" value="true"/>
    <parameter key="sample_size" value="-1"/>
    <list key="sample_size_per_class">
    <parameter key="true" value="%{exampleCount}"/>
    <parameter key="false" value="%{exampleCount}"/>
    </list>
    <list key="sample_ratio_per_class"/>
    <list key="sample_probability_per_class"/>
    </operator>
    <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (2)" width="90" x="380" y="34">
    <parameter key="criterion" value="gini_index"/>
    <parameter key="apply_prepruning" value="false"/>
    <parameter key="minimal_gain" value="0.01"/>
    <parameter key="minimal_leaf_size" value="4"/>
    </operator>
    <connect from_port="training set" to_op="Sample (2)" to_port="example set input"/>
    <connect from_op="Sample (2)" from_port="example set output" to_op="Decision Tree (2)" to_port="training set"/>
    <connect from_op="Decision Tree (2)" from_port="model" to_port="model"/>
    <portSpacing port="source_training set" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_through 1" spacing="0"/>
    </process>
    <process expanded="true">
    <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model (4)" width="90" x="112" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance (4)" width="90" x="246" y="34">
    <parameter key="main_criterion" value="recall"/>
    <parameter key="AUC (optimistic)" value="true"/>
    <parameter key="AUC" value="true"/>
    <parameter key="AUC (pessimistic)" value="true"/>
    <parameter key="precision" value="true"/>
    <parameter key="recall" value="true"/>
    </operator>
    <connect from_port="model" to_op="Apply Model (4)" to_port="model"/>
    <connect from_port="test set" to_op="Apply Model (4)" to_port="unlabelled data"/>
    <connect from_op="Apply Model (4)" from_port="labelled data" to_op="Performance (4)" to_port="labelled data"/>
    <connect from_op="Performance (4)" from_port="performance" to_port="performance 1"/>
    <portSpacing port="source_model" spacing="0"/>
    <portSpacing port="source_test set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="sink_test set results" spacing="0"/>
    <portSpacing port="sink_performance 1" spacing="0"/>
    <portSpacing port="sink_performance 2" spacing="0"/>
    </process>
    </operator>
    <connect from_port="input 1" to_op="Extract Macro (2)" to_port="example set"/>
    <connect from_op="Extract Macro (2)" from_port="example set" to_op="DT Cross Validation (2)" to_port="example set"/>
    <connect from_op="DT Cross Validation (2)" from_port="model" to_port="model"/>
    <connect from_op="DT Cross Validation (2)" from_port="performance 1" to_port="performance"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="source_input 2" spacing="0"/>
    <portSpacing port="sink_performance" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_output 1" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="8.2.000" expanded="true" height="124" name="kNN Optimize Parameters (2)" width="90" x="849" y="493">
    <list key="parameters">
    <parameter key="k-NN (2).k" value="[1.0;500;500;linear]"/>
    </list>
    <process expanded="true">
    <operator activated="true" class="extract_macro" compatibility="8.2.000" expanded="true" height="68" name="Extract Macro (3)" width="90" x="112" y="34">
    <parameter key="macro" value="exampleCount"/>
    <parameter key="macro_type" value="statistics"/>
    <parameter key="statistics" value="count"/>
    <parameter key="attribute_name" value="Donation"/>
    <parameter key="attribute_value" value="true"/>
    <list key="additional_macros"/>
    </operator>
    <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="DT Cross Validation (3)" width="90" x="581" y="34">
    <process expanded="true">
    <operator activated="true" class="sample" compatibility="8.2.000" expanded="true" height="82" name="Sample (3)" width="90" x="179" y="34">
    <parameter key="balance_data" value="true"/>
    <parameter key="sample_size" value="-1"/>
    <list key="sample_size_per_class">
    <parameter key="true" value="%{exampleCount}"/>
    <parameter key="false" value="%{exampleCount}"/>
    </list>
    <list key="sample_ratio_per_class"/>
    <list key="sample_probability_per_class"/>
    </operator>
    <operator activated="true" class="k_nn" compatibility="8.2.000" expanded="true" height="82" name="k-NN (2)" width="90" x="380" y="34">
    <parameter key="k" value="5"/>
    <parameter key="weighted_vote" value="true"/>
    </operator>
    <connect from_port="training set" to_op="Sample (3)" to_port="example set input"/>
    <connect from_op="Sample (3)" from_port="example set output" to_op="k-NN (2)" to_port="training set"/>
    <connect from_op="k-NN (2)" from_port="model" to_port="model"/>
    <portSpacing port="source_training set" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_through 1" spacing="0"/>
    </process>
    <process expanded="true">
    <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model (2)" width="90" x="112" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance (2)" width="90" x="246" y="34">
    <parameter key="main_criterion" value="recall"/>
    <parameter key="AUC (optimistic)" value="true"/>
    <parameter key="AUC" value="true"/>
    <parameter key="AUC (pessimistic)" value="true"/>
    <parameter key="precision" value="true"/>
    <parameter key="recall" value="true"/>
    </operator>
    <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
    <connect from_port="test set" to_op="Apply Model (2)" to_port="unlabelled data"/>
    <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
    <connect from_op="Performance (2)" from_port="performance" to_port="performance 1"/>
    <portSpacing port="source_model" spacing="0"/>
    <portSpacing port="source_test set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="sink_test set results" spacing="0"/>
    <portSpacing port="sink_performance 1" spacing="0"/>
    <portSpacing port="sink_performance 2" spacing="0"/>
    </process>
    </operator>
    <connect from_port="input 1" to_op="Extract Macro (3)" to_port="example set"/>
    <connect from_op="Extract Macro (3)" from_port="example set" to_op="DT Cross Validation (3)" to_port="example set"/>
    <connect from_op="DT Cross Validation (3)" from_port="model" to_port="model"/>
    <connect from_op="DT Cross Validation (3)" from_port="performance 1" to_port="performance"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="source_input 2" spacing="0"/>
    <portSpacing port="sink_performance" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_output 1" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="compare_rocs" compatibility="8.2.000" expanded="true" height="82" name="Compare ROCs (3)" width="90" x="849" y="34">
    <parameter key="sampling_type" value="automatic"/>
    <parameter key="use_example_weights" value="false"/>
    <process expanded="true">
    <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (5)" width="90" x="447" y="34"/>
    <operator activated="true" class="naive_bayes" compatibility="8.2.000" expanded="true" height="82" name="Naive Bayes (3)" width="90" x="447" y="136"/>
    <operator activated="true" class="rule_induction" compatibility="8.2.000" expanded="true" height="82" name="Rule Induction (3)" width="90" x="447" y="238"/>
    <operator activated="true" class="random_tree" compatibility="8.2.000" expanded="true" height="82" name="Random Tree (3)" width="90" x="447" y="340"/>
    <operator activated="true" class="k_nn" compatibility="8.2.000" expanded="true" height="82" name="k-NN (5)" width="90" x="447" y="442"/>
    <connect from_port="train 1" to_op="Decision Tree (5)" to_port="training set"/>
    <connect from_port="train 2" to_op="Naive Bayes (3)" to_port="training set"/>
    <connect from_port="train 3" to_op="Rule Induction (3)" to_port="training set"/>
    <connect from_port="train 4" to_op="Random Tree (3)" to_port="training set"/>
    <connect from_port="train 5" to_op="k-NN (5)" to_port="training set"/>
    <connect from_op="Decision Tree (5)" from_port="model" to_port="model 1"/>
    <connect from_op="Naive Bayes (3)" from_port="model" to_port="model 2"/>
    <connect from_op="Rule Induction (3)" from_port="model" to_port="model 3"/>
    <connect from_op="Random Tree (3)" from_port="model" to_port="model 4"/>
    <connect from_op="k-NN (5)" from_port="model" to_port="model 5"/>
    <portSpacing port="source_train 1" spacing="0"/>
    <portSpacing port="source_train 2" spacing="0"/>
    <portSpacing port="source_train 3" spacing="0"/>
    <portSpacing port="source_train 4" spacing="0"/>
    <portSpacing port="source_train 5" spacing="0"/>
    <portSpacing port="source_train 6" spacing="0"/>
    <portSpacing port="sink_model 1" spacing="0"/>
    <portSpacing port="sink_model 2" spacing="0"/>
    <portSpacing port="sink_model 3" spacing="0"/>
    <portSpacing port="sink_model 4" spacing="0"/>
    <portSpacing port="sink_model 5" spacing="0"/>
    <portSpacing port="sink_model 6" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="Cross Validation" width="90" x="849" y="136">
    <process expanded="true">
    <operator activated="true" class="extract_macro" compatibility="8.2.000" expanded="true" height="68" name="Extract Macro" width="90" x="112" y="34">
    <parameter key="macro" value="exampleCount"/>
    <parameter key="macro_type" value="statistics"/>
    <parameter key="statistics" value="count"/>
    <parameter key="attribute_name" value="Donation"/>
    <parameter key="attribute_value" value="true"/>
    <list key="additional_macros"/>
    </operator>
    <operator activated="true" class="sample" compatibility="8.2.000" expanded="true" height="82" name="Sample" width="90" x="246" y="34">
    <parameter key="balance_data" value="true"/>
    <parameter key="sample_size" value="-1"/>
    <list key="sample_size_per_class">
    <parameter key="true" value="%{exampleCount}"/>
    <parameter key="false" value="%{exampleCount}"/>
    </list>
    <list key="sample_ratio_per_class"/>
    <list key="sample_probability_per_class"/>
    </operator>
    <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree" width="90" x="380" y="34"/>
    <connect from_port="training set" to_op="Extract Macro" to_port="example set"/>
    <connect from_op="Extract Macro" from_port="example set" to_op="Sample" to_port="example set input"/>
    <connect from_op="Sample" from_port="example set output" to_op="Decision Tree" to_port="training set"/>
    <connect from_op="Decision Tree" from_port="model" to_port="model"/>
    <portSpacing port="source_training set" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_through 1" spacing="0"/>
    </process>
    <process expanded="true">
    <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model" width="90" x="112" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance" width="90" x="246" y="34">
    <parameter key="main_criterion" value="recall"/>
    <parameter key="precision" value="true"/>
    <parameter key="recall" value="true"/>
    </operator>
    <connect from_port="model" to_op="Apply Model" to_port="model"/>
    <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
    <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
    <connect from_op="Performance" from_port="performance" to_port="performance 1"/>
    <portSpacing port="source_model" spacing="0"/>
    <portSpacing port="source_test set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="sink_test set results" spacing="0"/>
    <portSpacing port="sink_performance 1" spacing="0"/>
    <portSpacing port="sink_performance 2" spacing="0"/>
    </process>
    </operator>
    <connect from_op="Read CSV" from_port="output" to_op="Rename" to_port="example set input"/>
    <connect from_op="Rename" from_port="example set output" to_op="Numerical to Binominal" to_port="example set input"/>
    <connect from_op="Numerical to Binominal" from_port="example set output" to_op="Set Role" to_port="example set input"/>
    <connect from_op="Set Role" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
    <connect from_op="Select Attributes" from_port="example set output" to_op="Multiply (2)" to_port="input"/>
    <connect from_op="Multiply (2)" from_port="output 1" to_op="Compare ROCs (3)" to_port="example set"/>
    <connect from_op="Multiply (2)" from_port="output 2" to_port="result 4"/>
    <connect from_op="Multiply (2)" from_port="output 3" to_op="kNN Optimize Parameters (2)" to_port="input 1"/>
    <connect from_op="Multiply (2)" from_port="output 4" to_op="Cross Validation" to_port="example set"/>
    <connect from_op="Multiply (2)" from_port="output 5" to_port="result 8"/>
    <connect from_op="Multiply (2)" from_port="output 6" to_op="DT Optimize Parameters" to_port="input 1"/>
    <connect from_op="DT Optimize Parameters" from_port="performance" to_port="result 9"/>
    <connect from_op="DT Optimize Parameters" from_port="model" to_port="result 10"/>
    <connect from_op="DT Optimize Parameters" from_port="parameter set" to_port="result 11"/>
    <connect from_op="kNN Optimize Parameters (2)" from_port="performance" to_port="result 5"/>
    <connect from_op="kNN Optimize Parameters (2)" from_port="model" to_port="result 6"/>
    <connect from_op="kNN Optimize Parameters (2)" from_port="parameter set" to_port="result 7"/>
    <connect from_op="Compare ROCs (3)" from_port="rocComparison" to_port="result 1"/>
    <connect from_op="Cross Validation" from_port="model" to_port="result 2"/>
    <connect from_op="Cross Validation" from_port="performance 1" to_port="result 3"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    <portSpacing port="sink_result 3" spacing="0"/>
    <portSpacing port="sink_result 4" spacing="0"/>
    <portSpacing port="sink_result 5" spacing="0"/>
    <portSpacing port="sink_result 6" spacing="0"/>
    <portSpacing port="sink_result 7" spacing="0"/>
    <portSpacing port="sink_result 8" spacing="0"/>
    <portSpacing port="sink_result 9" spacing="0"/>
    <portSpacing port="sink_result 10" spacing="0"/>
    <portSpacing port="sink_result 11" spacing="0"/>
    <portSpacing port="sink_result 12" spacing="0"/>
    </process>
    </operator>
    </process>

     

Answers

  • cam8cam8 Member Posts: 2 Contributor I

    Hey @lionelderkrikor,

     

    thank you for your support.

    I was already worried about the model.

    I test your solution with the macro- an the sample-operator.

    The accuracy increased instantly.

     

    With your solution i tested a few other things and see, that the cross validation was a important fact for decreasing the accuracy.

    Now I am working with the Optimize Parameters and directly below with your solution and the model without the cross validation.

     

    Again: Thank you very much.

    Best Regards

    car8

    sgenzer
Sign In or Register to comment.