🎉 🎉   RAPIDMINER 9.5 BETA IS OUT!!!   🎉 🎉



🦉 🎤   RapidMiner Wisdom 2020 - CALL FOR SPEAKERS   🦉 🎤

We are inviting all community members to submit proposals to speak at Wisdom 2020 in Boston.

Whether it's a cool RapidMiner trick or a use case implementation, we want to see what you have.
Form link is below and deadline for submissions is November 15. See you in Boston!


Reference Macro Value (from constructed macro name)

NoelNoel Member Posts: 77  Maven
edited October 4 in Help
Hi All (@lionelderkrikor )-

There's a 'Set Macros from Example set' operator. Like in the attached process (with excel source file), I'd like to feed macro names and values into the process and then, in a 'Loop Until' operator, access the value of a macro whose name I construct (from another macro).

So, as in the example below, I have a 'train_start_date1' attribute with a value. In the loop, I want to take the string 'train_start_date', stick the iteration number on the end (%{iteration}), form a macro name 'train_start_date1' and then have access to the value of that macro (namely 5/31/2016).

Any help appreciated! Thanks.


Best Answer

  • sgenzersgenzer 12Posts: 2,525  Community Manager
    Solution Accepted
    hi @Noel ok been scratching my head for 5 min trying to figure out what the problem is. 😀Is this what you're looking for?

    <?xml version="1.0" encoding="UTF-8"?><process version="9.4.001">
      <operator activated="true" class="process" compatibility="9.4.000" expanded="true" name="Process">
        <parameter key="logverbosity" value="init"/>
        <parameter key="random_seed" value="2001"/>
        <parameter key="send_mail" value="never"/>
        <parameter key="notification_email" value=""/>
        <parameter key="process_duration_for_mail" value="30"/>
        <parameter key="encoding" value="SYSTEM"/>
        <process expanded="true">
          <operator activated="true" class="read_excel" compatibility="9.4.001" expanded="true" height="68" name="Read Excel" width="90" x="45" y="34">
            <parameter key="excel_file" value="/Users/genzerconsulting/Desktop/DateConstructionTest.v2.xlsx"/>
            <parameter key="sheet_selection" value="sheet number"/>
            <parameter key="sheet_number" value="1"/>
            <parameter key="imported_cell_range" value="A1"/>
            <parameter key="encoding" value="SYSTEM"/>
            <parameter key="first_row_as_names" value="true"/>
            <list key="annotations"/>
            <parameter key="date_format" value=""/>
            <parameter key="time_zone" value="SYSTEM"/>
            <parameter key="locale" value="English (United States)"/>
            <parameter key="read_all_values_as_polynominal" value="false"/>
            <list key="data_set_meta_data_information">
              <parameter key="0" value="macro.true.polynominal.attribute"/>
              <parameter key="1" value="value.true.date.attribute"/>
            <parameter key="read_not_matching_values_as_missings" value="false"/>
            <parameter key="datamanagement" value="double_array"/>
            <parameter key="data_management" value="auto"/>
          <operator activated="true" class="concurrency:loop" compatibility="9.4.001" expanded="true" height="82" name="Loop" width="90" x="179" y="34">
            <parameter key="number_of_iterations" value="5"/>
            <parameter key="iteration_macro" value="iteration"/>
            <parameter key="reuse_results" value="false"/>
            <parameter key="enable_parallel_execution" value="false"/>
            <process expanded="true">
              <operator activated="true" breakpoints="after" class="generate_macro" compatibility="9.4.001" expanded="true" height="82" name="Generate Macro (2)" width="90" x="45" y="34">
                <list key="function_descriptions">
                  <parameter key="foo" value="concat(&quot;train_start_date&quot;,%{iteration})"/>
              <operator activated="true" breakpoints="after" class="extract_macro" compatibility="9.4.001" expanded="true" height="68" name="Extract Macro (3)" width="90" x="179" y="34">
                <parameter key="macro" value="%{foo}"/>
                <parameter key="macro_type" value="data_value"/>
                <parameter key="statistics" value="average"/>
                <parameter key="attribute_name" value="value"/>
                <parameter key="example_index" value="%{iteration}"/>
                <list key="additional_macros"/>
              <connect from_port="input 1" to_op="Generate Macro (2)" to_port="through 1"/>
              <connect from_op="Generate Macro (2)" from_port="through 1" to_op="Extract Macro (3)" to_port="example set"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="source_input 2" spacing="0"/>
              <portSpacing port="sink_output 1" spacing="0"/>
              <portSpacing port="sink_output 2" spacing="0"/>
          <connect from_op="Read Excel" from_port="output" to_op="Loop" to_port="input 1"/>
          <connect from_op="Loop" from_port="output 1" to_port="result 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>

    Don't forget to submit your great ideas for Wisdom 2020! Deadline is November 15.

    Wisdom 2020 – Call for Speakers Form 


  • kaymankayman Member Posts: 396   Unicorn
    One way would be to loop your examples, then extract macro to store your Excell fields as macro and then generate macro to alter these values for your final macro value. 

    Or you could generate a new attribute using your Excell data and then extract this new attribute using the extract macro operator using a loop. 
  • NoelNoel Member Posts: 77  Maven
    Hi @kayman -

    Thanks for your response. Did you have a chance to look at the process I uploaded, because I’m not sure what the difference is between what you suggested and what I was trying to do.

    The crux of the matter is that I want to be absolutely sure the macro values agree with the number of the iteration running. That’s why I was trying to concatenate the iteration number onto a string. I can’t figure out how to reference the value of the macro whose name is the same as the resulting string...
  • kaymankayman Member Posts: 396   Unicorn
    Ouch, I responded from my tablet in my lazy chair, so not able to check your stuff. At least we think alike :-)

    Looks like Scott is more active and has a solution but if not I definitely will have a closer look at it. 
  • NoelNoel Member Posts: 77  Maven
    No worries! I may have misunderstood what you wrote; I barely know up from down in this domain.
  • NoelNoel Member Posts: 77  Maven
    And I'm envious that you're in a lazy chair!
  • NoelNoel Member Posts: 77  Maven
    Hi Scott- Appreciate the help! You nailed it. Thanks, much.

  • NoelNoel Member Posts: 77  Maven
    Hi Scott / @sgenzer

    The specs for my request were probably wrong-headed. I.E. I was probably going about the task the wrong way.

    I tried using the macros and their values from your example in Filter Examples (process and input files attached). I'm wondering if that operator doesn't take macros for inputs or if I'm doing it wrong.

    To back up and give you some context, my desire to create a configurable, iterative training and testing process. I want to train a model on some span of time (that is passed as a parameter) and then test on another variable span of time (also passed as an input).The process would step forward by the amount of the test span and repeat the process.

    I probably did a terrible job explaining this. Let me know if I can provide further clarification.
  • NoelNoel Member Posts: 77  Maven
    Scott / @sgenzer -

    "Filter Example Range" seems to support macros, so I'll work with that. It'll also sidestep the date format problem I'm having.

    Thanks again for your help.
  • NoelNoel Member Posts: 77  Maven
    Scott / @sgenzer -

    How do I go about requesting that the "Filter Example" operator be enhanced to support macros?

    As I said above, I can work with the ...Range flavor, but that involves me converting dates to period numbers and it is just a matter of time until I make a mistake.

  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,525  Community Manager
    hi @Noel my apologies I think I dropped this thread. Thanks for pinging me again.

    You can use macros for Filter Examples - particularly if you use the "expression" parameter rather than the "custom filters".

    I'm a bit tangled on what you want to do now...can you re-clarify? :blush:

    Don't forget to submit your great ideas for Wisdom 2020! Deadline is November 15.

    Wisdom 2020 – Call for Speakers Form 

  • NoelNoel Member Posts: 77  Maven
    No worries!

    I'm doing iterative training and testing... given some contiguous dates (say a week), train for some period of time before the target week,  then test on the target week, and repeat. (I couldn't figure out how to deal with the macros and syntax/formatting of dates, so I used period numbers.)

    I have a fair amount of data and didn't want to manually run each week, so, in addition to my "real data", I input a table that looks like this:

              interation_number, train_start_period, train_end_period, test_start_period, test_end_period

    I put a row in the above table for each week I wanted to "run" and iterating over those rows did the repetitive training and testing programmaticallly (I wrote the output for each week to the database).

    I'm doing bound-to-fail conversions from dates (as I logically think about my data set) to period numbers in order to make my process work. It's just a matter of time before I mess something up and I won't catch the mistake for a while because a range of numbers means nothing to me visually:

    Optically, this is meaningful:

              5, 9/30/17, 9/30/19, 10/1/19, 10/4/19

    This, on the other hand, is not:


Sign In or Register to comment.