Options
"Deep learning extension : Questions and Warnings in the process"
lionelderkrikor
Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 
Unicorn
Hi all,
I'm using the new Deep Learning extension (TS to Tensor and Deep Learning (Tensor) operators) in a binary classification project.
The dataset is composed of a collection of 4 examples set. Each example set has 5 attributes (4 regular attributes + target) and 800000 rows.
The target has the same value (0 or 1) for a given example set :
1. When I'm executing the process, the process duration is around 5 hours ! ...( I have a laptop 4 x 2.5 GHz / 16 Go RAM / Windows 10).
My questions are:
2. As a workaround I'm using the Sample operator to reduce the number of rows from 800000 -> 1000 and in fine decrease significantly the computation time. But when executed, the process raises some "warnings" in the Log :
"Couldn't update the network in epoch 1"
"Couldn't update the network in epoch 2"
"Couldn't update the network in epoch 3"
.
"Couldn't update the network in epoch N"
Can you explain this behaviour ? What I have to do in my proces to avoid that ?
Regards,
Lionel
NB : the data :
- the 4 example sets (files signal_x_target_y) to store in a directory to set in the Loop Files operator :
https://drive.google.com/open?id=1tNHkk-N7HVivWmDKaByEIXyk_8UKZU3R
- the file metadata_train.csv to feed the Read CSV operator inside the Loop Collection operator :
https://drive.google.com/open?id=1oTXFeb60FfpjlG7b46aSIwfkcCe7NCZ_
NB2 : The process :
I'm using the new Deep Learning extension (TS to Tensor and Deep Learning (Tensor) operators) in a binary classification project.
The dataset is composed of a collection of 4 examples set. Each example set has 5 attributes (4 regular attributes + target) and 800000 rows.
The target has the same value (0 or 1) for a given example set :
1. When I'm executing the process, the process duration is around 5 hours ! ...( I have a laptop 4 x 2.5 GHz / 16 Go RAM / Windows 10).
My questions are:
- Is it the normal expected time ?
- If yes, what do you recommend me, knowing that I'm performing just preliminary tests (The whole dataset has in reality 2904 example sets !)
2. As a workaround I'm using the Sample operator to reduce the number of rows from 800000 -> 1000 and in fine decrease significantly the computation time. But when executed, the process raises some "warnings" in the Log :
"Couldn't update the network in epoch 1"
"Couldn't update the network in epoch 2"
"Couldn't update the network in epoch 3"
.
"Couldn't update the network in epoch N"
Can you explain this behaviour ? What I have to do in my proces to avoid that ?
Regards,
Lionel
NB : the data :
- the 4 example sets (files signal_x_target_y) to store in a directory to set in the Loop Files operator :
https://drive.google.com/open?id=1tNHkk-N7HVivWmDKaByEIXyk_8UKZU3R
- the file metadata_train.csv to feed the Read CSV operator inside the Loop Collection operator :
https://drive.google.com/open?id=1oTXFeb60FfpjlG7b46aSIwfkcCe7NCZ_
NB2 : The process :
<?xml version="1.0" encoding="UTF-8"?><process version="9.2.000-RC">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="9.2.000-RC" expanded="true" name="Process">
<parameter key="logverbosity" value="init"/>
<parameter key="random_seed" value="2001"/>
<parameter key="send_mail" value="never"/>
<parameter key="notification_email" value=""/>
<parameter key="process_duration_for_mail" value="30"/>
<parameter key="encoding" value="SYSTEM"/>
<process expanded="true">
<operator activated="true" class="concurrency:loop_files" compatibility="9.2.000-RC" expanded="true" height="82" name="Loop Files" width="90" x="45" y="85">
<parameter key="directory" value="D:\Lionel\Data_sciences\Competitions\all\Data"/>
<parameter key="filter_type" value="glob"/>
<parameter key="recursive" value="false"/>
<parameter key="enable_macros" value="false"/>
<parameter key="macro_for_file_name" value="file_name"/>
<parameter key="macro_for_file_type" value="file_type"/>
<parameter key="macro_for_folder_name" value="folder_name"/>
<parameter key="reuse_results" value="false"/>
<parameter key="enable_parallel_execution" value="true"/>
<process expanded="true">
<operator activated="true" class="read_csv" compatibility="9.2.000-RC" expanded="true" height="68" name="Read CSV" width="90" x="246" y="85">
<parameter key="column_separators" value=","/>
<parameter key="trim_lines" value="false"/>
<parameter key="use_quotes" value="true"/>
<parameter key="quotes_character" value="""/>
<parameter key="escape_character" value="\"/>
<parameter key="skip_comments" value="false"/>
<parameter key="comment_characters" value="#"/>
<parameter key="starting_row" value="1"/>
<parameter key="parse_numbers" value="true"/>
<parameter key="decimal_character" value="."/>
<parameter key="grouped_digits" value="false"/>
<parameter key="grouping_character" value=","/>
<parameter key="infinity_representation" value=""/>
<parameter key="date_format" value=""/>
<parameter key="first_row_as_names" value="true"/>
<list key="annotations"/>
<parameter key="time_zone" value="SYSTEM"/>
<parameter key="locale" value="English (United States)"/>
<parameter key="encoding" value="SYSTEM"/>
<parameter key="read_all_values_as_polynominal" value="false"/>
<list key="data_set_meta_data_information"/>
<parameter key="read_not_matching_values_as_missings" value="true"/>
<parameter key="datamanagement" value="double_array"/>
<parameter key="data_management" value="auto"/>
</operator>
<connect from_port="file object" to_op="Read CSV" to_port="file"/>
<connect from_op="Read CSV" from_port="output" to_port="output 1"/>
<portSpacing port="source_file object" spacing="0"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<operator activated="true" breakpoints="after" class="loop_collection" compatibility="9.2.000-RC" expanded="true" height="82" name="Loop Collection" width="90" x="179" y="85">
<parameter key="set_iteration_macro" value="true"/>
<parameter key="macro_name" value="iteration"/>
<parameter key="macro_start_value" value="0"/>
<parameter key="unfold" value="false"/>
<process expanded="true">
<operator activated="true" class="read_csv" compatibility="9.2.000-RC" expanded="true" height="68" name="Read CSV (2)" width="90" x="112" y="187">
<parameter key="csv_file" value="D:\Lionel\Data_sciences\Competitions\all\metadata_train.csv"/>
<parameter key="column_separators" value=","/>
<parameter key="trim_lines" value="false"/>
<parameter key="use_quotes" value="true"/>
<parameter key="quotes_character" value="""/>
<parameter key="escape_character" value="\"/>
<parameter key="skip_comments" value="false"/>
<parameter key="comment_characters" value="#"/>
<parameter key="starting_row" value="1"/>
<parameter key="parse_numbers" value="true"/>
<parameter key="decimal_character" value="."/>
<parameter key="grouped_digits" value="false"/>
<parameter key="grouping_character" value=","/>
<parameter key="infinity_representation" value=""/>
<parameter key="date_format" value=""/>
<parameter key="first_row_as_names" value="true"/>
<list key="annotations"/>
<parameter key="time_zone" value="SYSTEM"/>
<parameter key="locale" value="English (United States)"/>
<parameter key="encoding" value="SYSTEM"/>
<parameter key="read_all_values_as_polynominal" value="false"/>
<list key="data_set_meta_data_information"/>
<parameter key="read_not_matching_values_as_missings" value="true"/>
<parameter key="datamanagement" value="double_array"/>
<parameter key="data_management" value="auto"/>
</operator>
<operator activated="true" class="python_scripting:execute_python" compatibility="9.1.000" expanded="true" height="124" name="Execute Python" width="90" x="246" y="85">
<parameter key="script" value="import pandas as pd # rm_main is a mandatory function, # the number of arguments has to be the number of input ports (can be none) def rm_main(data,data_2): data['target'] = data_2['target'][3*%{iteration}] data.rm_metadata["target"]=(None,'label') # connect 2 output ports to see the results return data"/>
<parameter key="use_default_python" value="true"/>
<parameter key="package_manager" value="conda (anaconda)"/>
</operator>
<operator activated="true" class="rename_by_replacing" compatibility="9.2.000-RC" expanded="true" height="82" name="Rename by Replacing" width="90" x="514" y="85">
<parameter key="attribute_filter_type" value="regular_expression"/>
<parameter key="attribute" value=""/>
<parameter key="attributes" value=""/>
<parameter key="regular_expression" value="att.*"/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="attribute_value"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="time"/>
<parameter key="block_type" value="attribute_block"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_matrix_row_start"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="replace_what" value="att"/>
</operator>
<operator activated="true" class="concurrency:loop_attributes" compatibility="9.2.000-RC" expanded="true" height="82" name="Loop Attributes" width="90" x="648" y="85">
<parameter key="attribute_filter_type" value="regular_expression"/>
<parameter key="attribute" value=""/>
<parameter key="attributes" value=""/>
<parameter key="regular_expression" value="target"/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="attribute_value"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="time"/>
<parameter key="block_type" value="attribute_block"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_matrix_row_start"/>
<parameter key="invert_selection" value="true"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="attribute_name_macro" value="loop_attribute"/>
<parameter key="reuse_results" value="true"/>
<parameter key="enable_parallel_execution" value="true"/>
<process expanded="true">
<operator activated="true" class="branch" compatibility="9.2.000-RC" expanded="true" height="103" name="Branch" width="90" x="246" y="34">
<parameter key="condition_type" value="expression"/>
<parameter key="expression" value="eval(%{loop_attribute})%3==0 && %{loop_attribute}!=0 && %{loop_attribute}!=1&& %{loop_attribute}!=2"/>
<parameter key="io_object" value="ANOVAMatrix"/>
<parameter key="return_inner_output" value="true"/>
<process expanded="true">
<operator activated="true" class="rename_by_replacing" compatibility="9.2.000-RC" expanded="true" height="82" name="Rename by Replacing (2)" width="90" x="179" y="136">
<parameter key="attribute_filter_type" value="all"/>
<parameter key="attribute" value=""/>
<parameter key="attributes" value=""/>
<parameter key="regular_expression" value="0|1|2"/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="attribute_value"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="time"/>
<parameter key="block_type" value="attribute_block"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_matrix_row_start"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="replace_what" value="%{loop_attribute}"/>
<parameter key="replace_by" value="0"/>
</operator>
<connect from_port="condition" to_port="input 1"/>
<connect from_port="input 1" to_op="Rename by Replacing (2)" to_port="example set input"/>
<connect from_op="Rename by Replacing (2)" from_port="example set output" to_port="input 2"/>
<portSpacing port="source_condition" spacing="0"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_input 1" spacing="0"/>
<portSpacing port="sink_input 2" spacing="0"/>
<portSpacing port="sink_input 3" spacing="0"/>
</process>
<process expanded="true">
<connect from_port="condition" to_port="input 1"/>
<connect from_port="input 1" to_port="input 2"/>
<portSpacing port="source_condition" spacing="0"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_input 1" spacing="0"/>
<portSpacing port="sink_input 2" spacing="0"/>
<portSpacing port="sink_input 3" spacing="0"/>
</process>
</operator>
<operator activated="true" class="branch" compatibility="9.2.000-RC" expanded="true" height="103" name="Branch (2)" width="90" x="447" y="34">
<parameter key="condition_type" value="expression"/>
<parameter key="expression" value="eval(%{loop_attribute})%3==1 && %{loop_attribute}!=0 && %{loop_attribute}!=1&& %{loop_attribute}!=2"/>
<parameter key="io_object" value="ANOVAMatrix"/>
<parameter key="return_inner_output" value="true"/>
<process expanded="true">
<operator activated="true" class="rename_by_replacing" compatibility="9.2.000-RC" expanded="true" height="82" name="Rename by Replacing (3)" width="90" x="179" y="136">
<parameter key="attribute_filter_type" value="all"/>
<parameter key="attribute" value=""/>
<parameter key="attributes" value=""/>
<parameter key="regular_expression" value="0|1|2"/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="attribute_value"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="time"/>
<parameter key="block_type" value="attribute_block"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_matrix_row_start"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="replace_what" value="%{loop_attribute}"/>
<parameter key="replace_by" value="1"/>
</operator>
<connect from_port="condition" to_port="input 1"/>
<connect from_port="input 1" to_op="Rename by Replacing (3)" to_port="example set input"/>
<connect from_op="Rename by Replacing (3)" from_port="example set output" to_port="input 2"/>
<portSpacing port="source_condition" spacing="0"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_input 1" spacing="0"/>
<portSpacing port="sink_input 2" spacing="0"/>
<portSpacing port="sink_input 3" spacing="0"/>
</process>
<process expanded="true">
<connect from_port="condition" to_port="input 1"/>
<connect from_port="input 1" to_port="input 2"/>
<portSpacing port="source_condition" spacing="0"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_input 1" spacing="0"/>
<portSpacing port="sink_input 2" spacing="0"/>
<portSpacing port="sink_input 3" spacing="0"/>
</process>
</operator>
<operator activated="true" class="branch" compatibility="9.2.000-RC" expanded="true" height="103" name="Branch (3)" width="90" x="648" y="34">
<parameter key="condition_type" value="expression"/>
<parameter key="expression" value="eval(%{loop_attribute})%3==2 && %{loop_attribute}!=0 && %{loop_attribute}!=1&& %{loop_attribute}!=2"/>
<parameter key="io_object" value="ANOVAMatrix"/>
<parameter key="return_inner_output" value="true"/>
<process expanded="true">
<operator activated="true" class="rename_by_replacing" compatibility="9.2.000-RC" expanded="true" height="82" name="Rename by Replacing (4)" width="90" x="179" y="136">
<parameter key="attribute_filter_type" value="all"/>
<parameter key="attribute" value=""/>
<parameter key="attributes" value=""/>
<parameter key="regular_expression" value="0|1|2"/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="attribute_value"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="time"/>
<parameter key="block_type" value="attribute_block"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_matrix_row_start"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="replace_what" value="%{loop_attribute}"/>
<parameter key="replace_by" value="2"/>
</operator>
<connect from_port="condition" to_port="input 1"/>
<connect from_port="input 1" to_op="Rename by Replacing (4)" to_port="example set input"/>
<connect from_op="Rename by Replacing (4)" from_port="example set output" to_port="input 2"/>
<portSpacing port="source_condition" spacing="0"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_input 1" spacing="0"/>
<portSpacing port="sink_input 2" spacing="0"/>
<portSpacing port="sink_input 3" spacing="0"/>
</process>
<process expanded="true">
<connect from_port="condition" to_port="input 1"/>
<connect from_port="input 1" to_port="input 2"/>
<portSpacing port="source_condition" spacing="0"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_input 1" spacing="0"/>
<portSpacing port="sink_input 2" spacing="0"/>
<portSpacing port="sink_input 3" spacing="0"/>
</process>
</operator>
<connect from_port="input 1" to_op="Branch" to_port="input 1"/>
<connect from_op="Branch" from_port="input 2" to_op="Branch (2)" to_port="input 1"/>
<connect from_op="Branch (2)" from_port="input 2" to_op="Branch (3)" to_port="input 1"/>
<connect from_op="Branch (3)" from_port="input 2" to_port="output 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<operator activated="false" class="numerical_to_binominal" compatibility="9.2.000-RC" expanded="true" height="82" name="Numerical to Binominal" width="90" x="380" y="187">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="target"/>
<parameter key="attributes" value=""/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="numeric"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="real"/>
<parameter key="block_type" value="value_series"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_series_end"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="true"/>
<parameter key="min" value="0.0"/>
<parameter key="max" value="0.0"/>
</operator>
<operator activated="true" class="operator_toolbox:create_exampleset" compatibility="1.7.000" expanded="true" height="68" name="Create ExampleSet" width="90" x="514" y="340">
<parameter key="generator_type" value="numeric_series"/>
<parameter key="number_of_examples" value="800000"/>
<parameter key="use_stepsize" value="false"/>
<list key="function_descriptions"/>
<parameter key="add_id_attribute" value="false"/>
<list key="numeric_series_configuration">
<parameter key="time" value="linear.1\.0.800000\.0"/>
</list>
<list key="date_series_configuration"/>
<list key="date_series_configuration (interval)"/>
<parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/>
<parameter key="column_separator" value=","/>
<parameter key="parse_all_as_nominal" value="false"/>
<parameter key="decimal_point_character" value="."/>
<parameter key="trim_attribute_names" value="true"/>
</operator>
<operator activated="true" class="generate_id" compatibility="9.2.000-RC" expanded="true" height="82" name="Generate ID" width="90" x="648" y="340">
<parameter key="create_nominal_ids" value="false"/>
<parameter key="offset" value="0"/>
</operator>
<operator activated="true" class="generate_id" compatibility="9.2.000-RC" expanded="true" height="82" name="Generate ID (2)" width="90" x="782" y="85">
<parameter key="create_nominal_ids" value="false"/>
<parameter key="offset" value="0"/>
</operator>
<operator activated="true" class="concurrency:join" compatibility="9.2.000-RC" expanded="true" height="82" name="Join" width="90" x="849" y="238">
<parameter key="remove_double_attributes" value="true"/>
<parameter key="join_type" value="inner"/>
<parameter key="use_id_attribute_as_key" value="true"/>
<list key="key_attributes">
<parameter key="time" value="target"/>
</list>
<parameter key="keep_both_join_attributes" value="false"/>
</operator>
<operator activated="true" class="sample" compatibility="9.2.000-RC" expanded="true" height="82" name="Sample" width="90" x="916" y="85">
<parameter key="sample" value="absolute"/>
<parameter key="balance_data" value="false"/>
<parameter key="sample_size" value="1000"/>
<parameter key="sample_ratio" value="0.1"/>
<parameter key="sample_probability" value="0.1"/>
<list key="sample_size_per_class"/>
<list key="sample_ratio_per_class"/>
<list key="sample_probability_per_class"/>
<parameter key="use_local_random_seed" value="false"/>
<parameter key="local_random_seed" value="1992"/>
</operator>
<operator activated="true" class="sort" compatibility="9.2.000-RC" expanded="true" height="82" name="Sort" width="90" x="1050" y="85">
<parameter key="attribute_name" value="time"/>
<parameter key="sorting_direction" value="increasing"/>
</operator>
<connect from_port="single" to_op="Execute Python" to_port="input 1"/>
<connect from_op="Read CSV (2)" from_port="output" to_op="Execute Python" to_port="input 2"/>
<connect from_op="Execute Python" from_port="output 1" to_op="Rename by Replacing" to_port="example set input"/>
<connect from_op="Rename by Replacing" from_port="example set output" to_op="Loop Attributes" to_port="input 1"/>
<connect from_op="Loop Attributes" from_port="output 1" to_op="Generate ID (2)" to_port="example set input"/>
<connect from_op="Create ExampleSet" from_port="output" to_op="Generate ID" to_port="example set input"/>
<connect from_op="Generate ID" from_port="example set output" to_op="Join" to_port="left"/>
<connect from_op="Generate ID (2)" from_port="example set output" to_op="Join" to_port="right"/>
<connect from_op="Join" from_port="join" to_op="Sample" to_port="example set input"/>
<connect from_op="Sample" from_port="example set output" to_op="Sort" to_port="example set input"/>
<connect from_op="Sort" from_port="example set output" to_port="output 1"/>
<portSpacing port="source_single" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<operator activated="true" class="deeplearning:dl4j_timeseries_converter" compatibility="0.9.000" expanded="true" height="68" name="TimeSeries to Tensor" width="90" x="447" y="85"/>
<operator activated="true" class="deeplearning:dl4j_tensor_sequential_neural_network" compatibility="0.9.000" expanded="true" height="103" name="Deep Learning (Tensor)" width="90" x="581" y="85">
<parameter key="loss_function" value="Multiclass Cross Entropy (Classification)"/>
<parameter key="epochs" value="100"/>
<parameter key="use_miniBatch" value="false"/>
<parameter key="batch_size" value="32"/>
<parameter key="updater" value="Adam"/>
<parameter key="learning_rate" value="0.01"/>
<parameter key="momentum" value="0.9"/>
<parameter key="rho" value="0.95"/>
<parameter key="epsilon" value="1.0E-6"/>
<parameter key="beta1" value="0.9"/>
<parameter key="beta2" value="0.999"/>
<parameter key="RMSdecay" value="0.95"/>
<parameter key="weight_initialization" value="Normal"/>
<parameter key="bias_initialization" value="0.0"/>
<parameter key="use_regularization" value="false"/>
<parameter key="l1_strength" value="0.1"/>
<parameter key="l2_strength" value="0.1"/>
<parameter key="optimization_method" value="Stochastic Gradient Descent"/>
<parameter key="backpropagation" value="Standard"/>
<parameter key="backpropagation_length" value="50"/>
<parameter key="infer_input_shape" value="true"/>
<parameter key="network_type" value="Simple Neural Network"/>
<parameter key="log_each_epoch" value="true"/>
<parameter key="epochs_per_log" value="10"/>
<parameter key="use_local_random_seed" value="false"/>
<parameter key="local_random_seed" value="1992"/>
<process expanded="true">
<operator activated="true" class="deeplearning:dl4j_lstm_layer" compatibility="0.9.000" expanded="true" height="68" name="Add LSTM Layer" origin="GENERATED_SAMPLE" width="90" x="313" y="136">
<parameter key="neurons" value="20"/>
<parameter key="gate_activation" value="TanH"/>
<parameter key="forget_gate_bias_initialization" value="1.0"/>
<description align="center" color="transparent" colored="false" width="126">A long short term memory layer is used for keeping previous time-steps for current training decisions.</description>
</operator>
<operator activated="true" class="deeplearning:dl4j_dense_layer" compatibility="0.9.000" expanded="true" height="68" name="Add Fully-Connected Layer" origin="GENERATED_SAMPLE" width="90" x="581" y="136">
<parameter key="number_of_neurons" value="2"/>
<parameter key="activation_function" value="Softmax"/>
<parameter key="use_dropout" value="false"/>
<parameter key="dropout_rate" value="0.25"/>
<parameter key="overwrite_networks_weight_initialization" value="false"/>
<parameter key="weight_initialization" value="Normal"/>
<parameter key="overwrite_networks_bias_initialization" value="false"/>
<parameter key="bias_initialization" value="0.0"/>
<description align="center" color="transparent" colored="false" width="126">Using no activation function for this output layer, since a regression is performed.</description>
</operator>
<connect from_port="layerArchitecture" to_op="Add LSTM Layer" to_port="layerArchitecture"/>
<connect from_op="Add LSTM Layer" from_port="layerArchitecture" to_op="Add Fully-Connected Layer" to_port="layerArchitecture"/>
<connect from_op="Add Fully-Connected Layer" from_port="layerArchitecture" to_port="layerArchitecture"/>
<portSpacing port="source_layerArchitecture" spacing="0"/>
<portSpacing port="sink_layerArchitecture" spacing="0"/>
</process>
</operator>
<connect from_op="Loop Files" from_port="output 1" to_op="Loop Collection" to_port="collection"/>
<connect from_op="Loop Collection" from_port="output 1" to_op="TimeSeries to Tensor" to_port="collection"/>
<connect from_op="TimeSeries to Tensor" from_port="tensor" to_op="Deep Learning (Tensor)" to_port="training set"/>
<connect from_op="Deep Learning (Tensor)" from_port="model" to_port="result 1"/>
<connect from_op="Deep Learning (Tensor)" from_port="history" to_port="result 2"/>
<connect from_op="Deep Learning (Tensor)" from_port="tensor" to_port="result 3"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
<portSpacing port="sink_result 4" spacing="0"/>
</process>
</operator>
</process>
2
Best Answer
-
Options
pschlunder
Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 96 
RM Research
Hi @lionelderkrikor,when you are performing a classification on time-series data converted with the shipped operator, the assumption is made, that each row of the ExampleSet represents one time-step. Hence using 800k rows in one ExampleSet means that 800k time-steps have to be considered for each example (represented as an ExampleSet). Hence if you are using a recurrent layer like LSTM, it tries to optimize hidden states for all of these time-steps. This is very resource hungry. Try using the truncated option for backpropagation in the Deep Learning (Tensor) operator. This essentially uses a similar idea like batch processing but on time-steps. So setting something like 50 as a backpropagation length would result in only taking 50 time-steps for each training pass reducing the need for resources a lot.I can recommend this great post from the guys over at DL4J about truncated backprop. in recurrent networks: https://deeplearning4j.org/docs/latest/deeplearning4j-nn-recurrent#tbpttIf you're still experiencing long execution times please let me know.Regarding the warning about not being able to update the network, this might be caused due to the network not converging. You could try changing the networks starting parameters like weight initialization or bias initialization as well as the learning rate or other parameters of the updaters.Regards,Philipp
6
Answers
I thank you for taking the time to answer me.
In deed, the parameter backpropagation = Truncated (backpropagation length = 50) accelerates very significantly the process (5 hours -> 90 sec.).
Moreover I haven't anymore the "warnings" during the process.
But, I played with the different parameters and now, I have, all the time, a score = 0, for all epochs :
Epoch score
1 0
2 0
3 0
.
.
N 0
Can you explain this behaviour ?
Regards,
Lionel