NOTE: IF YOU WISH TO REPORT A NEW BUG, PLEASE POST A NEW QUESTION AND TAG AS "BUG REPORT". THANK YOU.

Bug report - Error while merging attribute

hervedelhallehervedelhalle Member Posts: 14 Contributor II
edited November 2019 in Product Feedback
Hello everybody,

I am trying to merge attributes in a process. Everything seems ok but i got an error "Process failed" while running this process.
If i do not merge the attribtutes from the subprocess "Execute Min_Max_Freq" and get the result of this process in a dedicated table, it works fine.
However, if i try to merge the attributes from this subproccess to the attributes of the other subproccesses, it doesn't work anymore.
Can anybody help me ?

The error is not clear for me. Here is the details of the error

  • Exception: java.lang.NullPointerException
  • Message: null
  • Stack trace:
  • sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  • sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  • sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  • java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  • java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
  • java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
  • com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext.collectResults(AbstractConcurrencyContext.java:206)
  • com.rapidminer.studio.concurrency.internal.StudioConcurrencyContext.collectResults(StudioConcurrencyContext.java:33)
  • com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext.call(AbstractConcurrencyContext.java:141)
  • com.rapidminer.studio.concurrency.internal.StudioConcurrencyContext.call(StudioConcurrencyContext.java:33)
  • com.rapidminer.Process.executeRootInPool(Process.java:1349)
  • com.rapidminer.Process.execute(Process.java:1314)
  • com.rapidminer.Process.run(Process.java:1291)
  • com.rapidminer.Process.run(Process.java:1177)
  • com.rapidminer.Process.run(Process.java:1130)
  • com.rapidminer.Process.run(Process.java:1125)
  • com.rapidminer.Process.run(Process.java:1115)
  • com.rapidminer.gui.ProcessThread.run(ProcessThread.java:65)

  • Cause
  • Exception: java.lang.NullPointerException
  • Message: null
  • Stack trace:

Here is xml process :

<?xml version="1.0" encoding="UTF-8"?><process version="9.3.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="9.3.001" expanded="true" name="Process">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
      <operator activated="true" class="retrieve" compatibility="9.3.001" expanded="true" height="68" name="Retrieve" width="90" x="45" y="34">
        <parameter key="repository_entry" value="//Lecture des fichiers texte/test_store/data_time_series"/>
      </operator>
      <operator activated="true" class="numerical_to_real" compatibility="9.3.001" expanded="true" height="82" name="Numerical to Real" width="90" x="179" y="34">
        <parameter key="attribute_filter_type" value="single"/>
        <parameter key="attribute" value="TIME"/>
        <parameter key="attributes" value=""/>
        <parameter key="use_except_expression" value="false"/>
        <parameter key="value_type" value="numeric"/>
        <parameter key="use_value_type_exception" value="false"/>
        <parameter key="except_value_type" value="real"/>
        <parameter key="block_type" value="value_series"/>
        <parameter key="use_block_type_exception" value="false"/>
        <parameter key="except_block_type" value="value_series_end"/>
        <parameter key="invert_selection" value="false"/>
        <parameter key="include_special_attributes" value="false"/>
      </operator>
      <operator activated="true" class="multiply" compatibility="9.3.001" expanded="true" height="124" name="Multiply" width="90" x="313" y="34"/>
      <operator activated="true" class="productivity:execute_process" compatibility="9.3.001" expanded="true" height="68" name="Execute Min_Max_Freq" width="90" x="447" y="697">
        <parameter key="process_location" value="Min_Max_Freq"/>
        <parameter key="use_input" value="true"/>
        <parameter key="store_output" value="false"/>
        <parameter key="propagate_metadata_recursively" value="true"/>
        <parameter key="cache_process" value="true"/>
        <list key="macros"/>
        <parameter key="fail_for_unknown_macros" value="true"/>
      </operator>
      <operator activated="true" class="productivity:execute_process" compatibility="9.3.001" expanded="true" height="68" name="Execute Derive_angle" width="90" x="447" y="595">
        <parameter key="process_location" value="Derive_angle"/>
        <parameter key="use_input" value="true"/>
        <parameter key="store_output" value="false"/>
        <parameter key="propagate_metadata_recursively" value="true"/>
        <parameter key="cache_process" value="true"/>
        <list key="macros"/>
        <parameter key="fail_for_unknown_macros" value="true"/>
      </operator>
      <operator activated="true" class="productivity:execute_process" compatibility="9.3.001" expanded="true" height="68" name="Execute 02 Compute moving average and extrema  MODIFIE" width="90" x="447" y="34">
        <parameter key="process_location" value="02 Compute moving average and extrema  MODIFIE"/>
        <parameter key="use_input" value="true"/>
        <parameter key="store_output" value="false"/>
        <parameter key="propagate_metadata_recursively" value="true"/>
        <parameter key="cache_process" value="true"/>
        <list key="macros"/>
        <parameter key="fail_for_unknown_macros" value="true"/>
      </operator>
      <operator activated="true" class="multiply" compatibility="9.3.001" expanded="true" height="145" name="Multiply (3)" width="90" x="581" y="34"/>
      <operator activated="true" class="store" compatibility="9.3.001" expanded="true" height="68" name="Store" width="90" x="581" y="238">
        <parameter key="repository_entry" value="test_store/data_time_series_post_pro"/>
      </operator>
      <operator activated="true" class="filter_examples" compatibility="9.3.001" expanded="true" height="103" name="Filter Examples (Phase 1) (3)" width="90" x="715" y="340">
        <parameter key="parameter_expression" value=""/>
        <parameter key="condition_class" value="custom_filters"/>
        <parameter key="invert_filter" value="false"/>
        <list key="filters_list">
          <parameter key="filters_entry_key" value="Phase.eq.11"/>
        </list>
        <parameter key="filters_logic_and" value="true"/>
        <parameter key="filters_check_metadata" value="true"/>
      </operator>
      <operator activated="true" class="productivity:execute_process" compatibility="9.3.001" expanded="true" height="68" name="Execute 1st_2nd_Ov - 1st_last_Und (3)" width="90" x="849" y="340">
        <parameter key="process_location" value="1st_2nd_Ov - 1st_last_Und"/>
        <parameter key="use_input" value="true"/>
        <parameter key="store_output" value="false"/>
        <parameter key="propagate_metadata_recursively" value="true"/>
        <parameter key="cache_process" value="true"/>
        <list key="macros"/>
        <parameter key="fail_for_unknown_macros" value="true"/>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="9.3.001" expanded="true" height="82" name="Select Attributes (3)" width="90" x="983" y="340">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attribute" value=""/>
        <parameter key="attributes" value="avg_force|F_Ov_1"/>
        <parameter key="use_except_expression" value="false"/>
        <parameter key="value_type" value="attribute_value"/>
        <parameter key="use_value_type_exception" value="false"/>
        <parameter key="except_value_type" value="time"/>
        <parameter key="block_type" value="attribute_block"/>
        <parameter key="use_block_type_exception" value="false"/>
        <parameter key="except_block_type" value="value_matrix_row_start"/>
        <parameter key="invert_selection" value="false"/>
        <parameter key="include_special_attributes" value="false"/>
      </operator>
      <operator activated="true" class="rename" compatibility="9.3.001" expanded="true" height="82" name="Rename (3)" width="90" x="1117" y="340">
        <parameter key="old_name" value="avg_force"/>
        <parameter key="new_name" value="F3_stable"/>
        <list key="rename_additional_attributes">
          <parameter key="F_Ov_1" value="F3_1"/>
        </list>
      </operator>
      <operator activated="true" class="filter_examples" compatibility="9.3.001" expanded="true" height="103" name="Filter Examples (Phase 1) (2)" width="90" x="715" y="187">
        <parameter key="parameter_expression" value=""/>
        <parameter key="condition_class" value="custom_filters"/>
        <parameter key="invert_filter" value="false"/>
        <list key="filters_list">
          <parameter key="filters_entry_key" value="Phase.eq.2"/>
        </list>
        <parameter key="filters_logic_and" value="true"/>
        <parameter key="filters_check_metadata" value="true"/>
      </operator>
      <operator activated="true" class="productivity:execute_process" compatibility="9.3.001" expanded="true" height="68" name="Execute 1st_2nd_Ov - 1st_last_Und (2)" width="90" x="849" y="187">
        <parameter key="process_location" value="1st_2nd_Ov - 1st_last_Und"/>
        <parameter key="use_input" value="true"/>
        <parameter key="store_output" value="false"/>
        <parameter key="propagate_metadata_recursively" value="true"/>
        <parameter key="cache_process" value="true"/>
        <list key="macros"/>
        <parameter key="fail_for_unknown_macros" value="true"/>
      </operator>
      <operator activated="true" class="filter_examples" compatibility="9.3.001" expanded="true" height="103" name="Filter Examples (Phase 1)" width="90" x="715" y="34">
        <parameter key="parameter_expression" value=""/>
        <parameter key="condition_class" value="custom_filters"/>
        <parameter key="invert_filter" value="false"/>
        <list key="filters_list">
          <parameter key="filters_entry_key" value="Phase.eq.1"/>
        </list>
        <parameter key="filters_logic_and" value="true"/>
        <parameter key="filters_check_metadata" value="true"/>
      </operator>
      <operator activated="true" class="productivity:execute_process" compatibility="9.3.001" expanded="true" height="68" name="Execute 1st_2nd_Ov - 1st_last_Und" width="90" x="849" y="34">
        <parameter key="process_location" value="1st_2nd_Ov - 1st_last_Und"/>
        <parameter key="use_input" value="true"/>
        <parameter key="store_output" value="false"/>
        <parameter key="propagate_metadata_recursively" value="true"/>
        <parameter key="cache_process" value="true"/>
        <list key="macros"/>
        <parameter key="fail_for_unknown_macros" value="true"/>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="9.3.001" expanded="true" height="82" name="Select Attributes" width="90" x="983" y="34">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attribute" value=""/>
        <parameter key="attributes" value="avg_force|F_Ov_1"/>
        <parameter key="use_except_expression" value="false"/>
        <parameter key="value_type" value="attribute_value"/>
        <parameter key="use_value_type_exception" value="false"/>
        <parameter key="except_value_type" value="time"/>
        <parameter key="block_type" value="attribute_block"/>
        <parameter key="use_block_type_exception" value="false"/>
        <parameter key="except_block_type" value="value_matrix_row_start"/>
        <parameter key="invert_selection" value="false"/>
        <parameter key="include_special_attributes" value="false"/>
      </operator>
      <operator activated="true" class="rename" compatibility="9.3.001" expanded="true" height="82" name="Rename" width="90" x="1117" y="34">
        <parameter key="old_name" value="avg_force"/>
        <parameter key="new_name" value="F1_stable"/>
        <list key="rename_additional_attributes">
          <parameter key="F_Ov_1" value="F1_1"/>
        </list>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="9.3.001" expanded="true" height="82" name="Select Attributes (2)" width="90" x="983" y="187">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attribute" value=""/>
        <parameter key="attributes" value="avg_force|F_Ov_1"/>
        <parameter key="use_except_expression" value="false"/>
        <parameter key="value_type" value="attribute_value"/>
        <parameter key="use_value_type_exception" value="false"/>
        <parameter key="except_value_type" value="time"/>
        <parameter key="block_type" value="attribute_block"/>
        <parameter key="use_block_type_exception" value="false"/>
        <parameter key="except_block_type" value="value_matrix_row_start"/>
        <parameter key="invert_selection" value="false"/>
        <parameter key="include_special_attributes" value="false"/>
      </operator>
      <operator activated="true" class="rename" compatibility="9.3.001" expanded="true" height="82" name="Rename (2)" width="90" x="1117" y="187">
        <parameter key="old_name" value="avg_force"/>
        <parameter key="new_name" value="F2_stable"/>
        <list key="rename_additional_attributes">
          <parameter key="F_Ov_1" value="F2_1"/>
        </list>
      </operator>
      <operator activated="true" class="operator_toolbox:merge" compatibility="2.1.000" expanded="true" height="103" name="Merge Attributes (2)" width="90" x="1117" y="646">
        <parameter key="handling_of_duplicate_attributes" value="rename"/>
        <parameter key="handling_of_special_attributes" value="keep_first_special_other_regular"/>
        <parameter key="handling_of_duplicate_annotations" value="rename"/>
      </operator>
      <operator activated="true" class="operator_toolbox:merge" compatibility="2.1.000" expanded="true" height="145" name="Merge Attributes" width="90" x="1385" y="187">
        <parameter key="handling_of_duplicate_attributes" value="rename"/>
        <parameter key="handling_of_special_attributes" value="keep_first_special_other_regular"/>
        <parameter key="handling_of_duplicate_annotations" value="rename"/>
      </operator>
      <connect from_op="Retrieve" from_port="output" to_op="Numerical to Real" to_port="example set input"/>
      <connect from_op="Numerical to Real" from_port="example set output" to_op="Multiply" to_port="input"/>
      <connect from_op="Multiply" from_port="output 1" to_op="Execute 02 Compute moving average and extrema  MODIFIE" to_port="input 1"/>
      <connect from_op="Multiply" from_port="output 2" to_op="Execute Derive_angle" to_port="input 1"/>
      <connect from_op="Multiply" from_port="output 3" to_op="Execute Min_Max_Freq" to_port="input 1"/>
      <connect from_op="Execute Min_Max_Freq" from_port="result 1" to_op="Merge Attributes (2)" to_port="example set 2"/>
      <connect from_op="Execute Derive_angle" from_port="result 1" to_op="Merge Attributes (2)" to_port="example set 1"/>
      <connect from_op="Execute 02 Compute moving average and extrema  MODIFIE" from_port="result 1" to_op="Multiply (3)" to_port="input"/>
      <connect from_op="Multiply (3)" from_port="output 1" to_op="Filter Examples (Phase 1)" to_port="example set input"/>
      <connect from_op="Multiply (3)" from_port="output 2" to_op="Filter Examples (Phase 1) (2)" to_port="example set input"/>
      <connect from_op="Multiply (3)" from_port="output 3" to_op="Filter Examples (Phase 1) (3)" to_port="example set input"/>
      <connect from_op="Multiply (3)" from_port="output 4" to_op="Store" to_port="input"/>
      <connect from_op="Filter Examples (Phase 1) (3)" from_port="example set output" to_op="Execute 1st_2nd_Ov - 1st_last_Und (3)" to_port="input 1"/>
      <connect from_op="Execute 1st_2nd_Ov - 1st_last_Und (3)" from_port="result 1" to_op="Select Attributes (3)" to_port="example set input"/>
      <connect from_op="Select Attributes (3)" from_port="example set output" to_op="Rename (3)" to_port="example set input"/>
      <connect from_op="Rename (3)" from_port="example set output" to_op="Merge Attributes" to_port="example set 3"/>
      <connect from_op="Filter Examples (Phase 1) (2)" from_port="example set output" to_op="Execute 1st_2nd_Ov - 1st_last_Und (2)" to_port="input 1"/>
      <connect from_op="Execute 1st_2nd_Ov - 1st_last_Und (2)" from_port="result 1" to_op="Select Attributes (2)" to_port="example set input"/>
      <connect from_op="Filter Examples (Phase 1)" from_port="example set output" to_op="Execute 1st_2nd_Ov - 1st_last_Und" to_port="input 1"/>
      <connect from_op="Execute 1st_2nd_Ov - 1st_last_Und" from_port="result 1" to_op="Select Attributes" to_port="example set input"/>
      <connect from_op="Select Attributes" from_port="example set output" to_op="Rename" to_port="example set input"/>
      <connect from_op="Rename" from_port="example set output" to_op="Merge Attributes" to_port="example set 1"/>
      <connect from_op="Select Attributes (2)" from_port="example set output" to_op="Rename (2)" to_port="example set input"/>
      <connect from_op="Rename (2)" from_port="example set output" to_op="Merge Attributes" to_port="example set 2"/>
      <connect from_op="Merge Attributes (2)" from_port="merged set" to_op="Merge Attributes" to_port="example set 4"/>
      <connect from_op="Merge Attributes" from_port="merged set" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>




0
0 votes

Sent to Engineering · Last Updated

FT-157

Comments

  • lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Hi @hervedelhalle,

    In order we can reproduce what you observe and understand what's going on, could you share : 
     - the differents subprocesses 
     - your data

    One starting point  : have you tried to inspect the examplesets which are entering in the Merge Atributes operator (by putting a breakpoint before) ?

    Regards,

    Lionel
  • hervedelhallehervedelhalle Member Posts: 14 Contributor II
    Hello,

    I will prepare a package with the data file and subprocesses as soon as possible.
    Yes, I have put a breakpoint and here are the two tables which are entering the Merge Attributes.



  • hervedelhallehervedelhalle Member Posts: 14 Contributor II
    Hello,

    Here are the different subprocesses and my data.
    I hope it is a correct format for you to help me. If not, please me help to export correctly data you want in order to help me.

    Best regards

    Hervé


  • lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    @hervedelhalle,

    OK, I'm able to reproduce the bug.
    It's very strange.
    I will investigate...

    Regards,

    Lionel
  • lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    @hervedelhalle,

    The bug is linked to the Merge Attributes operator in your subprocess  Min_Max_Freq.
    I deleted it . Now the Min_Max_Freq subprocesses has 4 outputs and theses 4 outputs are merged directly in the final Merge Attributes operators in the main process.

    In attached files the working processes.

    Regards,

    Lionel

    PS : I saw french attributes names : You are based in France ? (I'm from Marseille in France...) 


  • hervedelhallehervedelhalle Member Posts: 14 Contributor II

    Thank you for your reply. Unfortunately, i have still a similar error when i run your proccess ...

    Here is the error I get :

    • Exception: java.lang.NullPointerException
    • Message: null
    • Stack trace:
    • sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    • sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    • sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    • java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    • java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
    • java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
    • com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext.collectResults(AbstractConcurrencyContext.java:206)
    • com.rapidminer.studio.concurrency.internal.StudioConcurrencyContext.collectResults(StudioConcurrencyContext.java:33)
    • com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext.call(AbstractConcurrencyContext.java:141)
    • com.rapidminer.studio.concurrency.internal.StudioConcurrencyContext.call(StudioConcurrencyContext.java:33)
    • com.rapidminer.Process.executeRootInPool(Process.java:1349)
    • com.rapidminer.Process.execute(Process.java:1314)
    • com.rapidminer.Process.run(Process.java:1291)
    • com.rapidminer.Process.run(Process.java:1177)
    • com.rapidminer.Process.run(Process.java:1130)
    • com.rapidminer.Process.run(Process.java:1125)
    • com.rapidminer.Process.run(Process.java:1115)
    • com.rapidminer.gui.ProcessThread.run(ProcessThread.java:65)

    • Cause
    • Exception: java.lang.NullPointerException
    • Message: null
    • Stack trace:
    • com.rapidminer.extension.operator.blending.Merge.doWork(Merge.java:242)
    • com.rapidminer.operator.Operator.execute(Operator.java:1031)
    • com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
    • com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:812)
    • com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:807)
    • java.security.AccessController.doPrivileged(Native Method)
    • com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807)
    • com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:423)
    • com.rapidminer.operator.Operator.execute(Operator.java:1031)
    • com.rapidminer.Process.executeRoot(Process.java:1372)
    • com.rapidminer.Process.lambda$executeRootInPool$5(Process.java:1351)
    • com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext$AdaptedCallable.exec(AbstractConcurrencyContext.java:328)
    • java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    • java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    • java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    • java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
    P.S. : Not in France but in Belgium :)

  • hervedelhallehervedelhalle Member Posts: 14 Contributor II

    Finally it works with your files. I have made a mistake in testing your solution. My bad.

    But if I try to apply the same principle for my subprocess Derive_angle, i.e. merge atttributes in my main process and not in my subprocess, the main process fails again with same kind of error.

    You will find the modified main subprocess and the min_max_frq and derive_angle subprocesses I use.

    Could you please explain me why it behaves like this ?

    Best regards


  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist
    Tagging @tftemme , who is the expert on it.
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • tftemmetftemme Administrator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 164 RM Research
    Hi @hervedelhalle , @lionelderkrikor

    I can confirm the bug. It is connected to the renaming of the duplicate annotations and the fact that through the combination of different Merge operator we have a unusual setup of annotation keys. As your annotations are all only the original source, you can just change the 'handling of duplicate annotations' parameter to 'keep_only_first' and the process works.

    Will check the bug non-the less. Thanks for reporting.

    Best regards,
    Fabian

Sign In or Register to comment.