Feature Selection inside X-Validation; How to prevent that “Example sets are not compatible”?

a_ka_k Member Posts: 2 Contributor I
edited December 2018 in Help

Hi,

when trying to select the features for my text classification task inside the Cross-Validation operator (in order to avoid over-optimistic accuracy values as described in https://rapidminer.com/learn-right-way-validate-models-part-4-accidental-contamination/) I ran into the following problem:

The different number of features results in the following error message: “Example sets are not compatible”.
I guess this is related to the functionality of the X-Validation. I tried to transfer the same number of features from “Training” to the “Testing” section of the X-Validation by transfering the wordlist, however this didn’t work.

 

Any advice regarding this problem would be much appreciated!

 

(enclosed the failing XML process; paths to the text-directories were removed)

<?xml version="1.0" encoding="UTF-8"?><process version="8.0.001">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="8.0.001" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="text:process_document_from_file" compatibility="7.5.000" expanded="true" height="82" name="Process Documents from Files" width="90" x="45" y="34">
<list key="text_directories"/>
<parameter key="create_word_vector" value="false"/>
<parameter key="keep_text" value="true"/>
<process expanded="true">
<connect from_port="document" to_port="document 1"/>
<portSpacing port="source_document" spacing="0"/>
<portSpacing port="sink_document 1" spacing="0"/>
<portSpacing port="sink_document 2" spacing="0"/>
</process>
<description align="left" color="transparent" colored="false" width="126">read in documentes from text directories</description>
</operator>
<operator activated="true" class="set_role" compatibility="8.0.001" expanded="true" height="82" name="Set Role (5)" width="90" x="179" y="34">
<parameter key="attribute_name" value="text"/>
<list key="set_additional_roles"/>
<description align="left" color="transparent" colored="true" width="126">select attribute &amp;quot;text&amp;quot; for preprocessing&amp;quot;</description>
</operator>
<operator activated="true" class="text:process_document_from_data" compatibility="7.5.000" expanded="true" height="82" name="Process Documents from Data (4)" width="90" x="313" y="34">
<list key="specify_weights"/>
<process expanded="true">
<operator activated="true" class="text:tokenize" compatibility="7.5.000" expanded="true" height="68" name="Tokenize" width="90" x="112" y="34">
<parameter key="language" value="German"/>
</operator>
<operator activated="true" class="text:filter_stopwords_german" compatibility="7.5.000" expanded="true" height="68" name="Filter Stopwords" width="90" x="246" y="34"/>
<operator activated="true" class="text:stem_german" compatibility="7.5.000" expanded="true" height="68" name="Stem" width="90" x="380" y="34"/>
<operator activated="true" class="text:generate_n_grams_terms" compatibility="7.5.000" expanded="true" height="68" name="Generate n-Grams" width="90" x="514" y="34"/>
<connect from_port="document" to_op="Tokenize" to_port="document"/>
<connect from_op="Tokenize" from_port="document" to_op="Filter Stopwords" to_port="document"/>
<connect from_op="Filter Stopwords" from_port="document" to_op="Stem" to_port="document"/>
<connect from_op="Stem" from_port="document" to_op="Generate n-Grams" to_port="document"/>
<connect from_op="Generate n-Grams" from_port="document" to_port="document 1"/>
<portSpacing port="source_document" spacing="0"/>
<portSpacing port="sink_document 1" spacing="0"/>
<portSpacing port="sink_document 2" spacing="0"/>
</process>
<description align="left" color="transparent" colored="false" width="126">preprocessing (tokenization, etc.)</description>
</operator>
<operator activated="true" class="set_role" compatibility="8.0.001" expanded="true" height="82" name="Set Role (6)" width="90" x="447" y="34">
<parameter key="attribute_name" value="label"/>
<parameter key="target_role" value="label"/>
<list key="set_additional_roles"/>
<description align="left" color="transparent" colored="false" width="126">set label</description>
</operator>
<operator activated="true" class="concurrency:cross_validation" compatibility="8.0.001" expanded="true" height="145" name="Cross Validation (2)" width="90" x="581" y="34">
<parameter key="number_of_folds" value="2"/>
<parameter key="enable_parallel_execution" value="false"/>
<process expanded="true">
<operator activated="true" class="multiply" compatibility="8.0.001" expanded="true" height="103" name="Multiply" width="90" x="45" y="34"/>
<operator activated="true" class="optimize_selection_evolutionary" compatibility="8.0.001" expanded="true" height="103" name="Optimize Selection (Evolutionary)" width="90" x="179" y="187">
<parameter key="max_number_of_attributes" value="150"/>
<parameter key="maximum_number_of_generations" value="2"/>
<process expanded="true">
<operator activated="true" class="concurrency:cross_validation" compatibility="8.0.001" expanded="true" height="145" name="Cross Validation" width="90" x="45" y="34">
<parameter key="use_local_random_seed" value="true"/>
<process expanded="true">
<operator activated="true" class="k_nn" compatibility="8.0.001" expanded="true" height="82" name="k-NN" width="90" x="45" y="34">
<parameter key="k" value="5"/>
</operator>
<connect from_port="training set" to_op="k-NN" to_port="training set"/>
<connect from_op="k-NN" from_port="model" to_port="model"/>
<portSpacing port="source_training set" spacing="0"/>
<portSpacing port="sink_model" spacing="0"/>
<portSpacing port="sink_through 1" spacing="0"/>
</process>
<process expanded="true">
<operator activated="true" class="apply_model" compatibility="8.0.001" expanded="true" height="82" name="Apply Model" width="90" x="45" y="34">
<list key="application_parameters"/>
</operator>
<operator activated="true" class="performance_classification" compatibility="8.0.001" expanded="true" height="82" name="Performance" width="90" x="179" y="34">
<parameter key="accuracy" value="false"/>
<parameter key="classification_error" value="true"/>
<list key="class_weights"/>
</operator>
<connect from_port="model" to_op="Apply Model" to_port="model"/>
<connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
<connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
<connect from_op="Performance" from_port="performance" to_port="performance 1"/>
<connect from_op="Performance" from_port="example set" to_port="test set results"/>
<portSpacing port="source_model" spacing="0"/>
<portSpacing port="source_test set" spacing="0"/>
<portSpacing port="source_through 1" spacing="0"/>
<portSpacing port="sink_test set results" spacing="0"/>
<portSpacing port="sink_performance 1" spacing="0"/>
<portSpacing port="sink_performance 2" spacing="0"/>
</process>
</operator>
<connect from_port="example set" to_op="Cross Validation" to_port="example set"/>
<connect from_op="Cross Validation" from_port="performance 1" to_port="performance"/>
<portSpacing port="source_example set" spacing="0"/>
<portSpacing port="source_through 1" spacing="0"/>
<portSpacing port="sink_performance" spacing="0"/>
</process>
</operator>
<operator activated="true" class="select_by_weights" compatibility="8.0.001" expanded="true" height="103" name="Select by Weights" width="90" x="313" y="34">
<parameter key="weight_relation" value="top p%"/>
<parameter key="k" value="150"/>
</operator>
<operator activated="true" class="k_nn" compatibility="8.0.001" expanded="true" height="82" name="k-NN (2)" width="90" x="447" y="34">
<parameter key="k" value="5"/>
</operator>
<connect from_port="training set" to_op="Multiply" to_port="input"/>
<connect from_op="Multiply" from_port="output 1" to_op="Select by Weights" to_port="example set input"/>
<connect from_op="Multiply" from_port="output 2" to_op="Optimize Selection (Evolutionary)" to_port="example set in"/>
<connect from_op="Optimize Selection (Evolutionary)" from_port="weights" to_op="Select by Weights" to_port="weights"/>
<connect from_op="Select by Weights" from_port="example set output" to_op="k-NN (2)" to_port="training set"/>
<connect from_op="Select by Weights" from_port="weights" to_port="through 1"/>
<connect from_op="k-NN (2)" from_port="model" to_port="model"/>
<portSpacing port="source_training set" spacing="0"/>
<portSpacing port="sink_model" spacing="0"/>
<portSpacing port="sink_through 1" spacing="0"/>
<portSpacing port="sink_through 2" spacing="0"/>
</process>
<process expanded="true">
<operator activated="true" class="select_by_weights" compatibility="8.0.001" expanded="true" height="103" name="Select by Weights (2)" width="90" x="179" y="136">
<parameter key="weight_relation" value="top p%"/>
<parameter key="k" value="150"/>
</operator>
<operator activated="true" class="apply_model" compatibility="8.0.001" expanded="true" height="82" name="Apply Model (2)" width="90" x="313" y="34">
<list key="application_parameters"/>
</operator>
<operator activated="true" class="performance_classification" compatibility="8.0.001" expanded="true" height="82" name="Performance (2)" width="90" x="447" y="34">
<parameter key="accuracy" value="false"/>
<parameter key="classification_error" value="true"/>
<list key="class_weights"/>
</operator>
<connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
<connect from_port="test set" to_op="Select by Weights (2)" to_port="example set input"/>
<connect from_port="through 1" to_op="Select by Weights (2)" to_port="weights"/>
<connect from_op="Select by Weights (2)" from_port="example set output" to_op="Apply Model (2)" to_port="unlabelled data"/>
<connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
<connect from_op="Performance (2)" from_port="performance" to_port="performance 1"/>
<connect from_op="Performance (2)" from_port="example set" to_port="test set results"/>
<portSpacing port="source_model" spacing="0"/>
<portSpacing port="source_test set" spacing="0"/>
<portSpacing port="source_through 1" spacing="0"/>
<portSpacing port="source_through 2" spacing="0"/>
<portSpacing port="sink_test set results" spacing="0"/>
<portSpacing port="sink_performance 1" spacing="0"/>
<portSpacing port="sink_performance 2" spacing="0"/>
</process>
</operator>
<operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Model" width="90" x="916" y="34">
<parameter key="repository_entry" value="X-Validation_korrekt_model"/>
</operator>
<operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store DocumentTermMatrix" width="90" x="916" y="238">
<parameter key="repository_entry" value="X-Validation_korrekt_DocumentTermMatrix"/>
</operator>
<operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Performance" width="90" x="916" y="340">
<parameter key="repository_entry" value="X-Validation_korrekt_Performance"/>
</operator>
<operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Testdokumente" width="90" x="916" y="136">
<parameter key="repository_entry" value="X-Validation_korrekt_testdokumente"/>
</operator>
<connect from_op="Process Documents from Files" from_port="example set" to_op="Set Role (5)" to_port="example set input"/>
<connect from_op="Set Role (5)" from_port="example set output" to_op="Process Documents from Data (4)" to_port="example set"/>
<connect from_op="Process Documents from Data (4)" from_port="example set" to_op="Set Role (6)" to_port="example set input"/>
<connect from_op="Set Role (6)" from_port="example set output" to_op="Cross Validation (2)" to_port="example set"/>
<connect from_op="Cross Validation (2)" from_port="model" to_op="Store Model" to_port="input"/>
<connect from_op="Cross Validation (2)" from_port="example set" to_op="Store DocumentTermMatrix" to_port="input"/>
<connect from_op="Cross Validation (2)" from_port="test result set" to_op="Store Testdokumente" to_port="input"/>
<connect from_op="Cross Validation (2)" from_port="performance 1" to_op="Store Performance" to_port="input"/>
<connect from_op="Store Model" from_port="through" to_port="result 1"/>
<connect from_op="Store DocumentTermMatrix" from_port="through" to_port="result 3"/>
<connect from_op="Store Performance" from_port="through" to_port="result 4"/>
<connect from_op="Store Testdokumente" from_port="through" to_port="result 2"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
<portSpacing port="sink_result 4" spacing="0"/>
<portSpacing port="sink_result 5" spacing="0"/>
</process>
</operator>
</process>

 

Best Answer

  • lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Solution Accepted

    Hi again @a_k,

     

    I made a mistake in the order of the input connections of Superset operator in the previous process.

    Here the new process : 

    <?xml version="1.0" encoding="UTF-8"?><process version="8.0.001">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="8.0.001" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="text:process_document_from_file" compatibility="7.5.000" expanded="true" height="82" name="Process Documents from Files" width="90" x="45" y="34">
    <list key="text_directories">
    <parameter key="test" value="C:\Users\Lionel\Documents\Formations_DataScience\Rapidminer\Tests_Rapidminer\Test_text_mining"/>
    <parameter key="test_2" value="C:\Users\Lionel\Documents\Formations_DataScience\Rapidminer\Tests_Rapidminer\Test_text_mining_2"/>
    </list>
    <parameter key="create_word_vector" value="false"/>
    <parameter key="keep_text" value="true"/>
    <process expanded="true">
    <connect from_port="document" to_port="document 1"/>
    <portSpacing port="source_document" spacing="0"/>
    <portSpacing port="sink_document 1" spacing="0"/>
    <portSpacing port="sink_document 2" spacing="0"/>
    </process>
    <description align="left" color="transparent" colored="false" width="126">read in documentes from text directories</description>
    </operator>
    <operator activated="true" class="set_role" compatibility="8.0.001" expanded="true" height="82" name="Set Role (5)" width="90" x="179" y="34">
    <parameter key="attribute_name" value="text"/>
    <list key="set_additional_roles"/>
    <description align="left" color="transparent" colored="true" width="126">select attribute &amp;quot;text&amp;quot; for preprocessing&amp;quot;</description>
    </operator>
    <operator activated="true" class="text:process_document_from_data" compatibility="7.5.000" expanded="true" height="82" name="Process Documents from Data (4)" width="90" x="313" y="34">
    <list key="specify_weights"/>
    <process expanded="true">
    <operator activated="true" class="text:tokenize" compatibility="7.5.000" expanded="true" height="68" name="Tokenize" width="90" x="112" y="34">
    <parameter key="language" value="German"/>
    </operator>
    <operator activated="true" class="text:filter_stopwords_german" compatibility="7.5.000" expanded="true" height="68" name="Filter Stopwords" width="90" x="246" y="34"/>
    <operator activated="true" class="text:stem_german" compatibility="7.5.000" expanded="true" height="68" name="Stem" width="90" x="380" y="34"/>
    <operator activated="true" class="text:generate_n_grams_terms" compatibility="7.5.000" expanded="true" height="68" name="Generate n-Grams" width="90" x="514" y="34"/>
    <connect from_port="document" to_op="Tokenize" to_port="document"/>
    <connect from_op="Tokenize" from_port="document" to_op="Filter Stopwords" to_port="document"/>
    <connect from_op="Filter Stopwords" from_port="document" to_op="Stem" to_port="document"/>
    <connect from_op="Stem" from_port="document" to_op="Generate n-Grams" to_port="document"/>
    <connect from_op="Generate n-Grams" from_port="document" to_port="document 1"/>
    <portSpacing port="source_document" spacing="0"/>
    <portSpacing port="sink_document 1" spacing="0"/>
    <portSpacing port="sink_document 2" spacing="0"/>
    </process>
    <description align="left" color="transparent" colored="false" width="126">preprocessing (tokenization, etc.)</description>
    </operator>
    <operator activated="true" class="set_role" compatibility="8.0.001" expanded="true" height="82" name="Set Role (6)" width="90" x="447" y="34">
    <parameter key="attribute_name" value="label"/>
    <parameter key="target_role" value="label"/>
    <list key="set_additional_roles"/>
    <description align="left" color="transparent" colored="false" width="126">set label</description>
    </operator>
    <operator activated="true" class="concurrency:cross_validation" compatibility="8.0.001" expanded="true" height="145" name="Cross Validation (2)" width="90" x="715" y="34">
    <parameter key="number_of_folds" value="3"/>
    <parameter key="enable_parallel_execution" value="false"/>
    <process expanded="true">
    <operator activated="true" class="multiply" compatibility="8.0.001" expanded="true" height="103" name="Multiply" width="90" x="45" y="34"/>
    <operator activated="true" class="remember" compatibility="8.0.001" expanded="true" height="68" name="Remember" width="90" x="179" y="34">
    <parameter key="name" value="Text"/>
    </operator>
    <operator activated="true" class="optimize_selection_evolutionary" compatibility="8.0.001" expanded="true" height="103" name="Optimize Selection (Evolutionary)" width="90" x="179" y="187">
    <parameter key="max_number_of_attributes" value="150"/>
    <parameter key="maximum_number_of_generations" value="2"/>
    <process expanded="true">
    <operator activated="true" class="concurrency:cross_validation" compatibility="8.0.001" expanded="true" height="145" name="Cross Validation" width="90" x="45" y="34">
    <parameter key="number_of_folds" value="3"/>
    <parameter key="use_local_random_seed" value="true"/>
    <process expanded="true">
    <operator activated="true" class="k_nn" compatibility="8.0.001" expanded="true" height="82" name="k-NN" width="90" x="45" y="34">
    <parameter key="k" value="5"/>
    </operator>
    <connect from_port="training set" to_op="k-NN" to_port="training set"/>
    <connect from_op="k-NN" from_port="model" to_port="model"/>
    <connect from_op="k-NN" from_port="exampleSet" to_port="through 1"/>
    <portSpacing port="source_training set" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_through 1" spacing="0"/>
    <portSpacing port="sink_through 2" spacing="0"/>
    </process>
    <process expanded="true">
    <operator activated="true" class="superset" compatibility="8.0.001" expanded="true" height="82" name="Superset (2)" width="90" x="112" y="85"/>
    <operator activated="true" class="apply_model" compatibility="8.0.001" expanded="true" height="82" name="Apply Model" width="90" x="246" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_classification" compatibility="8.0.001" expanded="true" height="82" name="Performance" width="90" x="380" y="34">
    <parameter key="accuracy" value="false"/>
    <parameter key="classification_error" value="true"/>
    <list key="class_weights"/>
    </operator>
    <connect from_port="model" to_op="Apply Model" to_port="model"/>
    <connect from_port="test set" to_op="Superset (2)" to_port="example set 1"/>
    <connect from_port="through 1" to_op="Superset (2)" to_port="example set 2"/>
    <connect from_op="Superset (2)" from_port="superset 1" to_op="Apply Model" to_port="unlabelled data"/>
    <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
    <connect from_op="Performance" from_port="performance" to_port="performance 1"/>
    <connect from_op="Performance" from_port="example set" to_port="test set results"/>
    <portSpacing port="source_model" spacing="0"/>
    <portSpacing port="source_test set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="source_through 2" spacing="0"/>
    <portSpacing port="sink_test set results" spacing="0"/>
    <portSpacing port="sink_performance 1" spacing="0"/>
    <portSpacing port="sink_performance 2" spacing="0"/>
    </process>
    </operator>
    <connect from_port="example set" to_op="Cross Validation" to_port="example set"/>
    <connect from_op="Cross Validation" from_port="performance 1" to_port="performance"/>
    <portSpacing port="source_example set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="sink_performance" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="select_by_weights" compatibility="8.0.001" expanded="true" height="103" name="Select by Weights" width="90" x="313" y="34">
    <parameter key="weight_relation" value="top p%"/>
    <parameter key="k" value="150"/>
    </operator>
    <operator activated="true" class="k_nn" compatibility="8.0.001" expanded="true" height="82" name="k-NN (2)" width="90" x="380" y="187">
    <parameter key="k" value="5"/>
    </operator>
    <connect from_port="training set" to_op="Multiply" to_port="input"/>
    <connect from_op="Multiply" from_port="output 1" to_op="Remember" to_port="store"/>
    <connect from_op="Multiply" from_port="output 2" to_op="Optimize Selection (Evolutionary)" to_port="example set in"/>
    <connect from_op="Remember" from_port="stored" to_op="Select by Weights" to_port="example set input"/>
    <connect from_op="Optimize Selection (Evolutionary)" from_port="weights" to_op="Select by Weights" to_port="weights"/>
    <connect from_op="Select by Weights" from_port="example set output" to_op="k-NN (2)" to_port="training set"/>
    <connect from_op="Select by Weights" from_port="weights" to_port="through 1"/>
    <connect from_op="k-NN (2)" from_port="model" to_port="model"/>
    <portSpacing port="source_training set" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_through 1" spacing="0"/>
    <portSpacing port="sink_through 2" spacing="0"/>
    </process>
    <process expanded="true">
    <operator activated="true" class="select_by_weights" compatibility="8.0.001" expanded="true" height="103" name="Select by Weights (2)" width="90" x="45" y="187">
    <parameter key="weight_relation" value="top p%"/>
    <parameter key="k" value="150"/>
    </operator>
    <operator activated="true" class="recall" compatibility="8.0.001" expanded="true" height="68" name="Recall" width="90" x="179" y="289">
    <parameter key="name" value="Text"/>
    </operator>
    <operator activated="true" class="superset" compatibility="8.0.001" expanded="true" height="82" name="Superset" width="90" x="313" y="238">
    <parameter key="include_special_attributes" value="true"/>
    </operator>
    <operator activated="true" class="apply_model" compatibility="8.0.001" expanded="true" height="82" name="Apply Model (2)" width="90" x="313" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_classification" compatibility="8.0.001" expanded="true" height="82" name="Performance (2)" width="90" x="447" y="34">
    <parameter key="accuracy" value="false"/>
    <parameter key="classification_error" value="true"/>
    <list key="class_weights"/>
    </operator>
    <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
    <connect from_port="test set" to_op="Select by Weights (2)" to_port="example set input"/>
    <connect from_port="through 1" to_op="Select by Weights (2)" to_port="weights"/>
    <connect from_op="Select by Weights (2)" from_port="example set output" to_op="Superset" to_port="example set 1"/>
    <connect from_op="Recall" from_port="result" to_op="Superset" to_port="example set 2"/>
    <connect from_op="Superset" from_port="superset 1" to_op="Apply Model (2)" to_port="unlabelled data"/>
    <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
    <connect from_op="Performance (2)" from_port="performance" to_port="performance 1"/>
    <connect from_op="Performance (2)" from_port="example set" to_port="test set results"/>
    <portSpacing port="source_model" spacing="0"/>
    <portSpacing port="source_test set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="source_through 2" spacing="0"/>
    <portSpacing port="sink_test set results" spacing="0"/>
    <portSpacing port="sink_performance 1" spacing="0"/>
    <portSpacing port="sink_performance 2" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Model" width="90" x="916" y="34">
    <parameter key="repository_entry" value="X-Validation_korrekt_model"/>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store DocumentTermMatrix" width="90" x="916" y="238">
    <parameter key="repository_entry" value="X-Validation_korrekt_DocumentTermMatrix"/>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Performance" width="90" x="916" y="340">
    <parameter key="repository_entry" value="X-Validation_korrekt_Performance"/>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Testdokumente" width="90" x="916" y="136">
    <parameter key="repository_entry" value="X-Validation_korrekt_testdokumente"/>
    </operator>
    <connect from_op="Process Documents from Files" from_port="example set" to_op="Set Role (5)" to_port="example set input"/>
    <connect from_op="Set Role (5)" from_port="example set output" to_op="Process Documents from Data (4)" to_port="example set"/>
    <connect from_op="Process Documents from Data (4)" from_port="example set" to_op="Set Role (6)" to_port="example set input"/>
    <connect from_op="Set Role (6)" from_port="example set output" to_op="Cross Validation (2)" to_port="example set"/>
    <connect from_op="Cross Validation (2)" from_port="model" to_op="Store Model" to_port="input"/>
    <connect from_op="Cross Validation (2)" from_port="example set" to_op="Store DocumentTermMatrix" to_port="input"/>
    <connect from_op="Cross Validation (2)" from_port="test result set" to_op="Store Testdokumente" to_port="input"/>
    <connect from_op="Cross Validation (2)" from_port="performance 1" to_op="Store Performance" to_port="input"/>
    <connect from_op="Store Model" from_port="through" to_port="result 1"/>
    <connect from_op="Store DocumentTermMatrix" from_port="through" to_port="result 3"/>
    <connect from_op="Store Performance" from_port="through" to_port="result 4"/>
    <connect from_op="Store Testdokumente" from_port="through" to_port="result 2"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    <portSpacing port="sink_result 3" spacing="0"/>
    <portSpacing port="sink_result 4" spacing="0"/>
    <portSpacing port="sink_result 5" spacing="0"/>
    </process>
    </operator>
    </process>

    Regards,

     

    Lionel

Answers

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    so kudos to you for reading @IngoRM 's blog post :) Yes I would think that you are right - there are different numbers of features in the training and testing sets so Apply Model does not work. Not sure I can debug without the docs but maybe someone else has a clever idea.

     

    Scott

     

     

  • lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn

    Hi @a_k,

     

    I try an experimental solution with help of Recall/Remember operators and the Superset operator :

    Here the process : 

    <?xml version="1.0" encoding="UTF-8"?><process version="8.0.001">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="8.0.001" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="text:process_document_from_file" compatibility="7.5.000" expanded="true" height="82" name="Process Documents from Files" width="90" x="45" y="34">
    <list key="text_directories">
    <parameter key="test" value="C:\Users\Lionel\Documents\Formations_DataScience\Rapidminer\Tests_Rapidminer\Test_text_mining"/>
    <parameter key="test_2" value="C:\Users\Lionel\Documents\Formations_DataScience\Rapidminer\Tests_Rapidminer\Test_text_mining_2"/>
    </list>
    <parameter key="create_word_vector" value="false"/>
    <parameter key="keep_text" value="true"/>
    <process expanded="true">
    <connect from_port="document" to_port="document 1"/>
    <portSpacing port="source_document" spacing="0"/>
    <portSpacing port="sink_document 1" spacing="0"/>
    <portSpacing port="sink_document 2" spacing="0"/>
    </process>
    <description align="left" color="transparent" colored="false" width="126">read in documentes from text directories</description>
    </operator>
    <operator activated="true" class="set_role" compatibility="8.0.001" expanded="true" height="82" name="Set Role (5)" width="90" x="179" y="34">
    <parameter key="attribute_name" value="text"/>
    <list key="set_additional_roles"/>
    <description align="left" color="transparent" colored="true" width="126">select attribute &amp;quot;text&amp;quot; for preprocessing&amp;quot;</description>
    </operator>
    <operator activated="true" class="text:process_document_from_data" compatibility="7.5.000" expanded="true" height="82" name="Process Documents from Data (4)" width="90" x="313" y="34">
    <list key="specify_weights"/>
    <process expanded="true">
    <operator activated="true" class="text:tokenize" compatibility="7.5.000" expanded="true" height="68" name="Tokenize" width="90" x="112" y="34">
    <parameter key="language" value="German"/>
    </operator>
    <operator activated="true" class="text:filter_stopwords_german" compatibility="7.5.000" expanded="true" height="68" name="Filter Stopwords" width="90" x="246" y="34"/>
    <operator activated="true" class="text:stem_german" compatibility="7.5.000" expanded="true" height="68" name="Stem" width="90" x="380" y="34"/>
    <operator activated="true" class="text:generate_n_grams_terms" compatibility="7.5.000" expanded="true" height="68" name="Generate n-Grams" width="90" x="514" y="34"/>
    <connect from_port="document" to_op="Tokenize" to_port="document"/>
    <connect from_op="Tokenize" from_port="document" to_op="Filter Stopwords" to_port="document"/>
    <connect from_op="Filter Stopwords" from_port="document" to_op="Stem" to_port="document"/>
    <connect from_op="Stem" from_port="document" to_op="Generate n-Grams" to_port="document"/>
    <connect from_op="Generate n-Grams" from_port="document" to_port="document 1"/>
    <portSpacing port="source_document" spacing="0"/>
    <portSpacing port="sink_document 1" spacing="0"/>
    <portSpacing port="sink_document 2" spacing="0"/>
    </process>
    <description align="left" color="transparent" colored="false" width="126">preprocessing (tokenization, etc.)</description>
    </operator>
    <operator activated="true" class="set_role" compatibility="8.0.001" expanded="true" height="82" name="Set Role (6)" width="90" x="447" y="34">
    <parameter key="attribute_name" value="label"/>
    <parameter key="target_role" value="label"/>
    <list key="set_additional_roles"/>
    <description align="left" color="transparent" colored="false" width="126">set label</description>
    </operator>
    <operator activated="true" class="concurrency:cross_validation" compatibility="8.0.001" expanded="true" height="145" name="Cross Validation (2)" width="90" x="715" y="34">
    <parameter key="number_of_folds" value="3"/>
    <parameter key="enable_parallel_execution" value="false"/>
    <process expanded="true">
    <operator activated="true" class="multiply" compatibility="8.0.001" expanded="true" height="103" name="Multiply" width="90" x="45" y="34"/>
    <operator activated="true" class="remember" compatibility="8.0.001" expanded="true" height="68" name="Remember" width="90" x="179" y="34">
    <parameter key="name" value="Text"/>
    </operator>
    <operator activated="true" class="optimize_selection_evolutionary" compatibility="8.0.001" expanded="true" height="103" name="Optimize Selection (Evolutionary)" width="90" x="179" y="187">
    <parameter key="max_number_of_attributes" value="150"/>
    <parameter key="maximum_number_of_generations" value="2"/>
    <process expanded="true">
    <operator activated="true" class="concurrency:cross_validation" compatibility="8.0.001" expanded="true" height="145" name="Cross Validation" width="90" x="45" y="34">
    <parameter key="number_of_folds" value="3"/>
    <parameter key="use_local_random_seed" value="true"/>
    <process expanded="true">
    <operator activated="true" class="k_nn" compatibility="8.0.001" expanded="true" height="82" name="k-NN" width="90" x="45" y="34">
    <parameter key="k" value="5"/>
    </operator>
    <connect from_port="training set" to_op="k-NN" to_port="training set"/>
    <connect from_op="k-NN" from_port="model" to_port="model"/>
    <connect from_op="k-NN" from_port="exampleSet" to_port="through 1"/>
    <portSpacing port="source_training set" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_through 1" spacing="0"/>
    <portSpacing port="sink_through 2" spacing="0"/>
    </process>
    <process expanded="true">
    <operator activated="true" class="superset" compatibility="8.0.001" expanded="true" height="82" name="Superset (2)" width="90" x="112" y="85"/>
    <operator activated="true" class="apply_model" compatibility="8.0.001" expanded="true" height="82" name="Apply Model" width="90" x="246" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_classification" compatibility="8.0.001" expanded="true" height="82" name="Performance" width="90" x="380" y="34">
    <parameter key="accuracy" value="false"/>
    <parameter key="classification_error" value="true"/>
    <list key="class_weights"/>
    </operator>
    <connect from_port="model" to_op="Apply Model" to_port="model"/>
    <connect from_port="test set" to_op="Superset (2)" to_port="example set 1"/>
    <connect from_port="through 1" to_op="Superset (2)" to_port="example set 2"/>
    <connect from_op="Superset (2)" from_port="superset 1" to_op="Apply Model" to_port="unlabelled data"/>
    <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
    <connect from_op="Performance" from_port="performance" to_port="performance 1"/>
    <connect from_op="Performance" from_port="example set" to_port="test set results"/>
    <portSpacing port="source_model" spacing="0"/>
    <portSpacing port="source_test set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="source_through 2" spacing="0"/>
    <portSpacing port="sink_test set results" spacing="0"/>
    <portSpacing port="sink_performance 1" spacing="0"/>
    <portSpacing port="sink_performance 2" spacing="0"/>
    </process>
    </operator>
    <connect from_port="example set" to_op="Cross Validation" to_port="example set"/>
    <connect from_op="Cross Validation" from_port="performance 1" to_port="performance"/>
    <portSpacing port="source_example set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="sink_performance" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="select_by_weights" compatibility="8.0.001" expanded="true" height="103" name="Select by Weights" width="90" x="313" y="34">
    <parameter key="weight_relation" value="top p%"/>
    <parameter key="k" value="150"/>
    </operator>
    <operator activated="true" class="k_nn" compatibility="8.0.001" expanded="true" height="82" name="k-NN (2)" width="90" x="380" y="187">
    <parameter key="k" value="5"/>
    </operator>
    <connect from_port="training set" to_op="Multiply" to_port="input"/>
    <connect from_op="Multiply" from_port="output 1" to_op="Remember" to_port="store"/>
    <connect from_op="Multiply" from_port="output 2" to_op="Optimize Selection (Evolutionary)" to_port="example set in"/>
    <connect from_op="Remember" from_port="stored" to_op="Select by Weights" to_port="example set input"/>
    <connect from_op="Optimize Selection (Evolutionary)" from_port="weights" to_op="Select by Weights" to_port="weights"/>
    <connect from_op="Select by Weights" from_port="example set output" to_op="k-NN (2)" to_port="training set"/>
    <connect from_op="Select by Weights" from_port="weights" to_port="through 1"/>
    <connect from_op="k-NN (2)" from_port="model" to_port="model"/>
    <portSpacing port="source_training set" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_through 1" spacing="0"/>
    <portSpacing port="sink_through 2" spacing="0"/>
    </process>
    <process expanded="true">
    <operator activated="true" class="select_by_weights" compatibility="8.0.001" expanded="true" height="103" name="Select by Weights (2)" width="90" x="45" y="187">
    <parameter key="weight_relation" value="top p%"/>
    <parameter key="k" value="150"/>
    </operator>
    <operator activated="true" class="recall" compatibility="8.0.001" expanded="true" height="68" name="Recall" width="90" x="179" y="289">
    <parameter key="name" value="Text"/>
    </operator>
    <operator activated="true" class="superset" compatibility="8.0.001" expanded="true" height="82" name="Superset" width="90" x="313" y="238">
    <parameter key="include_special_attributes" value="true"/>
    </operator>
    <operator activated="true" class="apply_model" compatibility="8.0.001" expanded="true" height="82" name="Apply Model (2)" width="90" x="313" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_classification" compatibility="8.0.001" expanded="true" height="82" name="Performance (2)" width="90" x="447" y="34">
    <parameter key="accuracy" value="false"/>
    <parameter key="classification_error" value="true"/>
    <list key="class_weights"/>
    </operator>
    <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
    <connect from_port="test set" to_op="Select by Weights (2)" to_port="example set input"/>
    <connect from_port="through 1" to_op="Select by Weights (2)" to_port="weights"/>
    <connect from_op="Select by Weights (2)" from_port="example set output" to_op="Superset" to_port="example set 2"/>
    <connect from_op="Recall" from_port="result" to_op="Superset" to_port="example set 1"/>
    <connect from_op="Superset" from_port="superset 1" to_op="Apply Model (2)" to_port="unlabelled data"/>
    <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
    <connect from_op="Performance (2)" from_port="performance" to_port="performance 1"/>
    <connect from_op="Performance (2)" from_port="example set" to_port="test set results"/>
    <portSpacing port="source_model" spacing="0"/>
    <portSpacing port="source_test set" spacing="0"/>
    <portSpacing port="source_through 1" spacing="0"/>
    <portSpacing port="source_through 2" spacing="0"/>
    <portSpacing port="sink_test set results" spacing="0"/>
    <portSpacing port="sink_performance 1" spacing="0"/>
    <portSpacing port="sink_performance 2" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Model" width="90" x="916" y="34">
    <parameter key="repository_entry" value="X-Validation_korrekt_model"/>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store DocumentTermMatrix" width="90" x="916" y="238">
    <parameter key="repository_entry" value="X-Validation_korrekt_DocumentTermMatrix"/>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Performance" width="90" x="916" y="340">
    <parameter key="repository_entry" value="X-Validation_korrekt_Performance"/>
    </operator>
    <operator activated="true" class="store" compatibility="8.0.001" expanded="true" height="68" name="Store Testdokumente" width="90" x="916" y="136">
    <parameter key="repository_entry" value="X-Validation_korrekt_testdokumente"/>
    </operator>
    <connect from_op="Process Documents from Files" from_port="example set" to_op="Set Role (5)" to_port="example set input"/>
    <connect from_op="Set Role (5)" from_port="example set output" to_op="Process Documents from Data (4)" to_port="example set"/>
    <connect from_op="Process Documents from Data (4)" from_port="example set" to_op="Set Role (6)" to_port="example set input"/>
    <connect from_op="Set Role (6)" from_port="example set output" to_op="Cross Validation (2)" to_port="example set"/>
    <connect from_op="Cross Validation (2)" from_port="model" to_op="Store Model" to_port="input"/>
    <connect from_op="Cross Validation (2)" from_port="example set" to_op="Store DocumentTermMatrix" to_port="input"/>
    <connect from_op="Cross Validation (2)" from_port="test result set" to_op="Store Testdokumente" to_port="input"/>
    <connect from_op="Cross Validation (2)" from_port="performance 1" to_op="Store Performance" to_port="input"/>
    <connect from_op="Store Model" from_port="through" to_port="result 1"/>
    <connect from_op="Store DocumentTermMatrix" from_port="through" to_port="result 3"/>
    <connect from_op="Store Performance" from_port="through" to_port="result 4"/>
    <connect from_op="Store Testdokumente" from_port="through" to_port="result 2"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    <portSpacing port="sink_result 3" spacing="0"/>
    <portSpacing port="sink_result 4" spacing="0"/>
    <portSpacing port="sink_result 5" spacing="0"/>
    </process>
    </operator>
    </process>

    Does this modified process answer to your problem ?

     

    Regards,

     

    Lionel

     

  • a_ka_k Member Posts: 2 Contributor I

    Yes, indeed, this process solves my problem!
    Thank you so much for your quick assistance!

  • lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn

    HI,

     

    Thanks you to @JEdward. It inspire me for this solution (He provide a (relative)  similar solution in a previous topic)

     

    Regards;

     

    Lionel

     

     

Sign In or Register to comment.