Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Loop Data
I am trying to create multiple data sets each current data set as to split this by the number of months defined by a generate attribute attribute i have labeled "term" which works out the length of a contract and i want to expand this dataset having values that will be the same per a contract split across the entire contract period showing each month separately ie 2020/03/01-2021/02/28.
I have tried to create a loop but im not sure i have set it up correctly.
Please could anyone assist me.
I have tried to create a loop but im not sure i have set it up correctly.
Please could anyone assist me.
<operator activated="true" class="generate_attributes" compatibility="9.10.000" expanded="true" height="82" name="Generate Attributes (13)" width="90" x="4534" y="544">
<list key="function_descriptions">
<parameter key="TERM" value="if(date_get(BEGIN_COV_DT,DATE_UNIT_YEAR)==(date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_YEAR) - 1),date_get(BEGIN_COV_DT,DATE_UNIT_MONTH)-(date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_MONTH)+1),if(date_get(BEGIN_COV_DT,DATE_UNIT_YEAR)==(date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_YEAR)),date_get(BEGIN_COV_DT,DATE_UNIT_MONTH)+(11-date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_MONTH)),""))"/>
</list>
<parameter key="keep_all" value="true"/>
</operator>
<operator activated="false" class="loop_examples" compatibility="9.10.000" expanded="true" height="82" name="Loop Examples" width="90" x="4400" y="442">
<parameter key="iteration_macro" value="example"/>
<process expanded="true">
<operator activated="false" class="concurrency:loop_attributes" compatibility="9.10.000" expanded="true" height="82" name="Loop Attributes (2)" width="90" x="514" y="187">
<parameter key="attribute_filter_type" value="all"/>
<parameter key="attribute" value=""/>
<parameter key="attributes" value=""/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="attribute_value"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="time"/>
<parameter key="block_type" value="attribute_block"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_matrix_row_start"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="attribute_name_macro" value="loop_attribute"/>
<parameter key="reuse_results" value="false"/>
<parameter key="enable_parallel_execution" value="true"/>
<process expanded="true">
<operator activated="true" class="generate_attributes" compatibility="9.10.000" expanded="true" height="82" name="Generate Attributes (7)" width="90" x="179" y="34">
<list key="function_descriptions">
<parameter key="End_Cov_Date" value="date_add(date_parse_custom(BEGIN_COV_DT,+1,DATE_UNIT_MONTH))"/>
</list>
<parameter key="keep_all" value="true"/>
</operator>
<connect from_port="input 1" to_op="Generate Attributes (7)" to_port="example set input"/>
<connect from_op="Generate Attributes (7)" from_port="example set output" to_port="output 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<operator activated="true" class="filter_examples" compatibility="9.10.000" expanded="true" height="103" name="Filter Examples (4)" width="90" x="179" y="34">
<parameter key="parameter_expression" value=""/>
<parameter key="condition_class" value="custom_filters"/>
<parameter key="invert_filter" value="false"/>
<list key="filters_list"/>
<parameter key="filters_logic_and" value="true"/>
<parameter key="filters_check_metadata" value="true"/>
</operator>
<operator activated="true" class="concurrency:loop_values" compatibility="9.10.000" expanded="true" height="82" name="Loop Values" width="90" x="313" y="34">
<parameter key="attribute" value="TERM"/>
<parameter key="iteration_macro" value="loop_value"/>
<parameter key="reuse_results" value="false"/>
<parameter key="enable_parallel_execution" value="true"/>
<process expanded="true">
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<connect from_port="example set" to_op="Filter Examples (4)" to_port="example set input"/>
<connect from_op="Filter Examples (4)" from_port="example set output" to_op="Loop Values" to_port="input 1"/>
<connect from_op="Loop Values" from_port="output 1" to_port="example set"/>
<portSpacing port="source_example set" spacing="0"/>
<portSpacing port="sink_example set" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
</process>
</operator>
<operator activated="true" class="order_attributes" compatibility="9.10.000" expanded="true" height="82" name="Reorder Attributes (18)" width="90" x="4668" y="544">
<parameter key="sort_mode" value="user specified"/>
<parameter key="attribute_ordering" value="REPORTING_DT|ENTITY_ID|INSURANCE_CONTRACT_ID|COVERAGE_CD|CEDED_FLG|CASHFLOW_TYPE_CD|CURRENCY_CD|INCURRED_CLAIM_DT|CASHFLOW_DT|ACQ_COSTS_AMT|ADM_COSTS_AMT|CLAIMS_AMT|ENTITY_FEE_AMT|GUAR_FV_CHNG_AMT|GUAR_FV_AMT|GUAR_PMT_AMT|PREM_AMT|PROVENUES_AMT|RE_COMM_AMT|RE_DEF_AMT|RSK_ADJ_AMT|SETTL_COSTS_AMT|UNDERL_FV_AMT|UNDERL_FV_CHNG_AMT|INVESTMENT_AMT"/>
<parameter key="use_regular_expressions" value="false"/>
<parameter key="handle_unmatched" value="append"/>
<parameter key="sort_direction" value="ascending"/>
</operator>
<operator activated="true" breakpoints="after" class="concurrency:loop_attributes" compatibility="9.10.000" expanded="true" height="82" name="Loop Attributes" width="90" x="4802" y="544">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="TERM"/>
<parameter key="attributes" value=""/>
<parameter key="use_except_expression" value="false"/>
<parameter key="value_type" value="attribute_value"/>
<parameter key="use_value_type_exception" value="false"/>
<parameter key="except_value_type" value="time"/>
<parameter key="block_type" value="attribute_block"/>
<parameter key="use_block_type_exception" value="false"/>
<parameter key="except_block_type" value="value_matrix_row_start"/>
<parameter key="invert_selection" value="false"/>
<parameter key="include_special_attributes" value="false"/>
<parameter key="attribute_name_macro" value="loop_attribute"/>
<parameter key="reuse_results" value="true"/>
<parameter key="enable_parallel_execution" value="true"/>
<process expanded="true">
<operator activated="true" class="filter_examples" compatibility="9.10.000" expanded="true" height="103" name="Filter Examples (5)" width="90" x="45" y="34">
<parameter key="parameter_expression" value=""/>
<parameter key="condition_class" value="custom_filters"/>
<parameter key="invert_filter" value="false"/>
<list key="filters_list">
<parameter key="filters_entry_key" value="INSURANCE_CONTRACT_ID.equals.%{loop_attributes}"/>
</list>
<parameter key="filters_logic_and" value="true"/>
<parameter key="filters_check_metadata" value="true"/>
</operator>
<operator activated="true" class="concurrency:loop_values" compatibility="9.10.000" expanded="true" height="82" name="Loop Values (4)" width="90" x="246" y="34">
<parameter key="attribute" value="END_COV_DT"/>
<parameter key="iteration_macro" value="loop_value"/>
<parameter key="reuse_results" value="true"/>
<parameter key="enable_parallel_execution" value="false"/>
<process expanded="true">
<operator activated="true" class="generate_attributes" compatibility="9.10.000" expanded="true" height="82" name="Generate Attributes (8)" width="90" x="246" y="34">
<list key="function_descriptions">
<parameter key="End_Cov_Date" value="date_add(BEGIN_COV_DT,+1,DATE_UNIT_MONTH)"/>
</list>
<parameter key="keep_all" value="true"/>
</operator>
<connect from_port="input 1" to_op="Generate Attributes (8)" to_port="example set input"/>
<connect from_op="Generate Attributes (8)" from_port="example set output" to_port="output 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<connect from_port="input 1" to_op="Filter Examples (5)" to_port="example set input"/>
<connect from_op="Filter Examples (5)" from_port="example set output" to_op="Loop Values (4)" to_port="input 1"/>
<connect from_op="Loop Values (4)" from_port="output 1" to_port="output 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
0
Answers
Assume 2 dates, BCD is 2020-11-20 and RD is 2021/01/09 this part of the function is triggered :
date_get(date_parse_custom(BEGIN_COV_DT,"yyyy-MM-dd"),DATE_UNIT_YEAR)==(date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_YEAR) - 1)
and the provided result would be
date_get(date_parse_custom(BEGIN_COV_DT,"yyyy-MM-dd"),DATE_UNIT_MONTH)-(date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_MONTH)+1)
or in this example 11 - (1+1) = 11 while it should be 2 (or 3 depending on how you deal with started months)
So wouldn't it be better to work with datediff instead?
You convert first both BCD and RD to proper dates using the nominal to date operator, and then use the date_diff operator. This gives you the difference in milliseconds so you can then say that one month is (in average) around 2629800000 ms so dividing the date_diff with the amount of ms in a month gives you the difference in months.
I've attached an example, if I understood wrong feel free to correct.
Thank you for your assistance, i have have managed to correct the term and it was easier to use the date_diff and convert the milliseconds to months, so for the loop in the process i want the loop to create a number of iterations based on the term so ie the term is 3 so for that CONTRACT_iD needs to be replicated 3 times but its been creating more iterations than the term specified as well as the loop needs to produce one file and its currently creating individual files per a CONTRACT_ID and i have tried to append the results but that has not worked, im not too sure why this is happening.
Please could you assist?
Kind Regards
Next, also provide some more data or simplified examples showing what you want to do and what fails. This could be dummy partial processes where you add (partial) data exports so people can see, recreate and try to support with solutions. It is just very hard to try to do this based on just a little explanation and no domain knowledge.