The Altair Community is migrating to a new platform to provide a better experience for you. The RapidMiner Community will merge with the Altair Community at the same time. In preparation for the migration, both communities are on read-only mode from July 15th - July 24th, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here.
Options

Does Windowing operator accept a macro for "window size"?

vgpoweredvgpowered Member Posts: 30 Contributor II
Hello,
Is it possible to use a macro in the Windowing  for "window size" parameter?

I have read the thread "Forecasting process - doing it right?" and it seems the operator can do, but I have the error "Required parameter missing"

What happens?

Best Answer

  • Options
    ceaperezceaperez Member Posts: 541 Unicorn
    Solution Accepted
    Hi @vgpowered

    here is the XML:

    <?xml version="1.0" encoding="UTF-8"?><process version="10.3.001">
    
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="10.3.001" expanded="true" name="Process" origin="GENERATED_TUTORIAL">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
    <operator activated="true" class="retrieve" compatibility="10.3.001" expanded="true" height="68" name="Retrieve Lake Huron" origin="GENERATED_TUTORIAL" width="90" x="45" y="34">
    <parameter key="repository_entry" value="//Samples/Time Series/data sets/Lake Huron"/>
    </operator>
    <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="10.3.001" expanded="true" height="124" name="Optimize Parameters (Grid)" width="90" x="246" y="34">
    <list key="parameters">
    <parameter key="Set Macro (Real).value" value="[2;10;5;linear]"/>
    </list>
    <parameter key="error_handling" value="fail on error"/>
    <parameter key="log_performance" value="true"/>
    <parameter key="log_all_criteria" value="false"/>
    <parameter key="synchronize" value="false"/>
    <parameter key="enable_parallel_execution" value="true"/>
    <process expanded="true">
    <operator activated="true" class="operator_toolbox:set_macro_real" compatibility="2.17.000" expanded="true" height="82" name="Set Macro (Real)" width="90" x="45" y="187">
    <parameter key="macro" value="uno"/>
    <parameter key="value" value="3.5"/>
    </operator>
    <operator activated="true" class="generate_macro" compatibility="10.3.001" expanded="true" height="82" name="Generate Macro" width="90" x="179" y="238">
    <list key="function_descriptions">
    <parameter key="dos" value="floor(eval(%{uno}))"/>
    </list>
    </operator>
    <operator activated="true" class="time_series:windowing" compatibility="10.3.001" expanded="true" height="103" name="Windowing (Series)" origin="GENERATED_TUTORIAL" width="90" x="179" y="34">
    <parameter key="attribute_filter_type" value="single"/>
    <parameter key="attribute" value="Lake surface level / feet"/>
    <parameter key="attributes" value=""/>
    <parameter key="use_except_expression" value="false"/>
    <parameter key="value_type" value="numeric"/>
    <parameter key="use_value_type_exception" value="false"/>
    <parameter key="except_value_type" value="real"/>
    <parameter key="block_type" value="value_series"/>
    <parameter key="use_block_type_exception" value="false"/>
    <parameter key="except_block_type" value="value_series_end"/>
    <parameter key="invert_selection" value="false"/>
    <parameter key="include_special_attributes" value="false"/>
    <parameter key="has_indices" value="true"/>
    <parameter key="indices_attribute" value="Date"/>
    <parameter key="sort_time_series" value="false"/>
    <parameter key="expert_settings" value="false"/>
    <parameter key="unit" value="example based"/>
    <parameter key="windows_defined" value="from start"/>
    <parameter key="custom_start_point" value="5"/>
    <parameter key="custom_end_point" value="100"/>
    <parameter key="window_size" value="%{dos}"/>
    <parameter key="custom_start_time" value="2000-01-01 00:00:00"/>
    <parameter key="custom_end_time" value="2030-01-01 00:00:00"/>
    <parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/>
    <parameter key="window_size_time" value="1.Hours"/>
    <parameter key="windows_stop_definition" value="from next window start"/>
    <parameter key="window_start_attribute" value=""/>
    <parameter key="window_stop_attribute" value=""/>
    <parameter key="no_overlapping_windows" value="false"/>
    <parameter key="step_size" value="1"/>
    <parameter key="step_size_time" value="1.Minutes"/>
    <parameter key="create_horizon_(labels)" value="true"/>
    <parameter key="horizon_attribute" value="Lake surface level / feet"/>
    <parameter key="horizon_size" value="1"/>
    <parameter key="horizon_offset" value="0"/>
    <parameter key="horizon_size_time" value="1.Hours"/>
    <parameter key="horizon_offset_time" value="0.Minutes"/>
    <parameter key="horizon_start_attribute" value=""/>
    <parameter key="horizon_stop_attribute" value=""/>
    <parameter key="empty_window_handling" value="add empty exampleset"/>
    </operator>
    <operator activated="true" class="h2o:deep_learning" compatibility="10.3.001" expanded="true" height="103" name="Deep Learning" width="90" x="313" y="34">
    <parameter key="activation" value="Rectifier"/>
    <enumeration key="hidden_layer_sizes">
    <parameter key="hidden_layer_sizes" value="50"/>
    <parameter key="hidden_layer_sizes" value="50"/>
    </enumeration>
    <enumeration key="hidden_dropout_ratios"/>
    <parameter key="reproducible_(uses_1_thread)" value="false"/>
    <parameter key="use_local_random_seed" value="false"/>
    <parameter key="local_random_seed" value="1992"/>
    <parameter key="epochs" value="10.0"/>
    <parameter key="compute_variable_importances" value="false"/>
    <parameter key="train_samples_per_iteration" value="-2"/>
    <parameter key="adaptive_rate" value="true"/>
    <parameter key="epsilon" value="1.0E-8"/>
    <parameter key="rho" value="0.99"/>
    <parameter key="learning_rate" value="0.005"/>
    <parameter key="learning_rate_annealing" value="1.0E-6"/>
    <parameter key="learning_rate_decay" value="1.0"/>
    <parameter key="momentum_start" value="0.0"/>
    <parameter key="momentum_ramp" value="1000000.0"/>
    <parameter key="momentum_stable" value="0.0"/>
    <parameter key="nesterov_accelerated_gradient" value="true"/>
    <parameter key="standardize" value="true"/>
    <parameter key="L1" value="1.0E-5"/>
    <parameter key="L2" value="0.0"/>
    <parameter key="max_w2" value="10.0"/>
    <parameter key="loss_function" value="Automatic"/>
    <parameter key="distribution_function" value="AUTO"/>
    <parameter key="early_stopping" value="false"/>
    <parameter key="stopping_rounds" value="1"/>
    <parameter key="stopping_metric" value="AUTO"/>
    <parameter key="stopping_tolerance" value="0.001"/>
    <parameter key="missing_values_handling" value="MeanImputation"/>
    <parameter key="max_runtime_seconds" value="0"/>
    <list key="expert_parameters"/>
    <list key="expert_parameters_"/>
    </operator>
    <operator activated="true" class="apply_model" compatibility="10.3.001" expanded="true" height="82" name="Apply Model" width="90" x="447" y="34">
    <list key="application_parameters"/>
    </operator>
    <operator activated="true" class="performance_regression" compatibility="10.3.001" expanded="true" height="82" name="Performance (2)" width="90" x="581" y="34">
    <parameter key="main_criterion" value="first"/>
    <parameter key="root_mean_squared_error" value="true"/>
    <parameter key="absolute_error" value="false"/>
    <parameter key="relative_error" value="false"/>
    <parameter key="relative_error_lenient" value="false"/>
    <parameter key="relative_error_strict" value="false"/>
    <parameter key="normalized_absolute_error" value="false"/>
    <parameter key="root_relative_squared_error" value="false"/>
    <parameter key="squared_error" value="false"/>
    <parameter key="correlation" value="false"/>
    <parameter key="squared_correlation" value="false"/>
    <parameter key="prediction_average" value="false"/>
    <parameter key="spearman_rho" value="false"/>
    <parameter key="kendall_tau" value="false"/>
    <parameter key="skip_undefined_labels" value="true"/>
    <parameter key="use_example_weights" value="true"/>
    </operator>
    <connect from_port="input 1" to_op="Windowing (Series)" to_port="example set"/>
    <connect from_op="Set Macro (Real)" from_port="through 1" to_op="Generate Macro" to_port="through 1"/>
    <connect from_op="Windowing (Series)" from_port="windowed example set" to_op="Deep Learning" to_port="training set"/>
    <connect from_op="Deep Learning" from_port="model" to_op="Apply Model" to_port="model"/>
    <connect from_op="Deep Learning" from_port="exampleSet" to_op="Apply Model" to_port="unlabelled data"/>
    <connect from_op="Apply Model" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
    <connect from_op="Performance (2)" from_port="performance" to_port="performance"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="source_input 2" spacing="0"/>
    <portSpacing port="sink_performance" spacing="0"/>
    <portSpacing port="sink_model" spacing="0"/>
    <portSpacing port="sink_output 1" spacing="0"/>
    </process>
    </operator>
    <connect from_op="Retrieve Lake Huron" from_port="output" to_op="Optimize Parameters (Grid)" to_port="input 1"/>
    <connect from_op="Optimize Parameters (Grid)" from_port="performance" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>

Answers

  • Options
    ceaperezceaperez Member Posts: 541 Unicorn
    Hi @vgpowered
    yes, is possible, please find attached a simple sample using a modification of the windows size operator tutorial.

    best, 

    Cesar
  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Hi Cesar,
    Sorry, Where is the attachment?
    By the way I'm using Windowing operator inside a Optimize parameter (Grid)...
  • Options
    ceaperezceaperez Member Posts: 541 Unicorn
    Hi @vgpowered
    The attached file is in the bottom of my message.

    Can you share your process? 

    Best, 

    Cesar 
  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Hi Cesar,
    I can only see an icon (it is a page with the upper right corner folded). When I move the mouse over this icon the "Macrowinsize.rmp" message appear, but nothing else. I think you have attached an image, not a file.

    My process is very complex, but only a draft (Accepts my apology)

    Víctor
  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Hi Cesar
    Maybe the Rapidminer Web is not working. I'm testing my own message and it shows somthing similar like yours (about attached files)...

  • Options
    ceaperezceaperez Member Posts: 541 Unicorn
    Hi @vgpowered
    I can download your process. Anyway this is the xml 

    <?xml version="1.0" encoding="UTF-8"?><process version="10.3.001">
    
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="10.3.001" expanded="true" name="Process" origin="GENERATED_TUTORIAL">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
    <operator activated="true" class="retrieve" compatibility="10.3.001" expanded="true" height="68" name="Retrieve Lake Huron" origin="GENERATED_TUTORIAL" width="90" x="112" y="34">
    <parameter key="repository_entry" value="//Samples/Time Series/data sets/Lake Huron"/>
    </operator>
    <operator activated="true" class="set_macro" compatibility="10.3.001" expanded="true" height="82" name="Set Macro" width="90" x="112" y="238">
    <parameter key="macro" value="Winsiz"/>
    <parameter key="value" value="4"/>
    </operator>
    <operator activated="true" class="time_series:process_windows" compatibility="10.3.001" expanded="true" height="103" name="Process Windows" origin="GENERATED_TUTORIAL" width="90" x="313" y="34">
    <parameter key="attribute_filter_type" value="all"/>
    <parameter key="attribute" value=""/>
    <parameter key="attributes" value=""/>
    <parameter key="use_except_expression" value="false"/>
    <parameter key="value_type" value="numeric"/>
    <parameter key="use_value_type_exception" value="false"/>
    <parameter key="except_value_type" value="real"/>
    <parameter key="block_type" value="value_series"/>
    <parameter key="use_block_type_exception" value="false"/>
    <parameter key="except_block_type" value="value_series_end"/>
    <parameter key="invert_selection" value="false"/>
    <parameter key="include_special_attributes" value="false"/>
    <parameter key="has_indices" value="true"/>
    <parameter key="indices_attribute" value="Date"/>
    <parameter key="sort_time_series" value="false"/>
    <parameter key="expert_settings" value="false"/>
    <parameter key="unit" value="example based"/>
    <parameter key="windows_defined" value="from start"/>
    <parameter key="custom_start_point" value="5"/>
    <parameter key="custom_end_point" value="100"/>
    <parameter key="window_size" value="%{Winsiz}"/>
    <parameter key="custom_start_time" value="2000-01-01 00:00:00"/>
    <parameter key="custom_end_time" value="2030-01-01 00:00:00"/>
    <parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/>
    <parameter key="window_size_time" value="1.Hours"/>
    <parameter key="windows_stop_definition" value="from next window start"/>
    <parameter key="window_start_attribute" value=""/>
    <parameter key="window_stop_attribute" value=""/>
    <parameter key="no_overlapping_windows" value="false"/>
    <parameter key="step_size" value="1"/>
    <parameter key="step_size_time" value="1.Minutes"/>
    <parameter key="create_horizon_(labels)" value="true"/>
    <parameter key="horizon_attribute" value="Lake surface level / feet"/>
    <parameter key="horizon_size" value="1"/>
    <parameter key="horizon_offset" value="0"/>
    <parameter key="horizon_size_time" value="1.Hours"/>
    <parameter key="horizon_offset_time" value="0.Minutes"/>
    <parameter key="horizon_start_attribute" value=""/>
    <parameter key="horizon_stop_attribute" value=""/>
    <parameter key="empty_window_handling" value="add empty exampleset"/>
    <parameter key="add_last_index_in_window_attribute" value="true"/>
    <parameter key="enable_parallel_execution" value="true"/>
    <process expanded="true">
    <operator activated="true" class="time_series:extract_std_descriptive_features" compatibility="10.3.001" expanded="true" height="82" name="Extract Aggregates" origin="GENERATED_TUTORIAL" width="90" x="313" y="34">
    <parameter key="attribute_filter_type" value="all"/>
    <parameter key="attribute" value=""/>
    <parameter key="attributes" value=""/>
    <parameter key="use_except_expression" value="false"/>
    <parameter key="value_type" value="numeric"/>
    <parameter key="use_value_type_exception" value="false"/>
    <parameter key="except_value_type" value="real"/>
    <parameter key="block_type" value="value_series"/>
    <parameter key="use_block_type_exception" value="false"/>
    <parameter key="except_block_type" value="value_series_end"/>
    <parameter key="invert_selection" value="false"/>
    <parameter key="include_special_attributes" value="false"/>
    <parameter key="sum" value="true"/>
    <parameter key="mean" value="true"/>
    <parameter key="geometric_mean" value="true"/>
    <parameter key="first_quartile" value="true"/>
    <parameter key="median" value="true"/>
    <parameter key="third_quartile" value="true"/>
    <parameter key="min" value="true"/>
    <parameter key="max" value="true"/>
    <parameter key="std_deviation" value="true"/>
    <parameter key="kurtosis" value="true"/>
    <parameter key="skewness" value="true"/>
    <parameter key="add_time_series_name" value="false"/>
    <parameter key="ignore_invalid_values" value="false"/>
    </operator>
    <connect from_port="windowed example set" to_op="Extract Aggregates" to_port="example set"/>
    <connect from_op="Extract Aggregates" from_port="features" to_port="output 1"/>
    <portSpacing port="source_windowed example set" spacing="0"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_output 1" spacing="0"/>
    <portSpacing port="sink_output 2" spacing="0"/>
    </process>
    </operator>
    <operator activated="true" class="append" compatibility="10.3.001" expanded="true" height="82" name="Append" origin="GENERATED_TUTORIAL" width="90" x="514" y="34">
    <parameter key="datamanagement" value="double_array"/>
    <parameter key="data_management" value="auto"/>
    <parameter key="merge_type" value="all"/>
    </operator>
    <connect from_op="Retrieve Lake Huron" from_port="output" to_op="Process Windows" to_port="example set"/>
    <connect from_op="Set Macro" from_port="through 1" to_op="Process Windows" to_port="custom windows"/>
    <connect from_op="Process Windows" from_port="output 1" to_op="Append" to_port="example set 1"/>
    <connect from_op="Append" from_port="merged set" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>

  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Thank Cesar,
    I think the problem is the Optimization operator.
    I have build a little example that demostrates there is a limitation.
    Do not lose your time with the previous complex procces, simply test the next example.

    Víctor
  • Options
    ceaperezceaperez Member Posts: 541 Unicorn
    Hola Victor,
    Just put the macro outside of the optimization operator.

    Saludos, 

    Cesar

  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Hi Cesar,
    But in this case you cannot optimize its value. You could say, you can optimize it directly, but in my real process there are several windowing operators (inside the Optimize process) that share this same value but not others. This is the issue.

    Víctor
  • Options
    ceaperezceaperez Member Posts: 541 Unicorn
    edited March 23
    Hi @vgpowered
    I have a possible solution:

    Best,

    César
  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Hi Cesar,
    The attachment does not work.

    Regards
    Víctor
  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Hi Cesar,
    Í cannot download the "windowing with optimiza.rmp" . Icon doesn't work,

    Víctor
  • Options
    vgpoweredvgpowered Member Posts: 30 Contributor II
    Hello César,
    Yes, this is the solution. I did'n have installed the Operator Toolbox set of operators. In all cases this is a real case of trick solutión. I think Ripidminer has a problem if we have to use this approach.

    It should be more efficient to create a new versión of the optimize operator.

    Thank you very much.

    Víctor
  • Options
    ceaperezceaperez Member Posts: 541 Unicorn
    Hi @vgpowered,
    You are right, this is a tricky solution, but these kinds of issues help the RM team improve the product. I'm not part of the RM team so we need help from someone there to review this issue or find other way, maybe @rjones13 can follow up this issue. 

    best 

    Cesar
Sign In or Register to comment.