"Read file depending on Macro-Value"

CrazyTCrazyT Member Posts: 8 Contributor II
edited June 2019 in Help
Hi,

is it possible to set the parameter "file name" of the "Read CSV"-operator to the content of a macro.
For example i want to check if different files exist(file1.csv, file2.csv, file3.csv ...) and then want to import them into one table of another database.
Tagged:

Answers

  • cherokeecherokee Member Posts: 82 Maven
    Hi CrazyT,

    yes it is possible to set the filename of any read operator with a macro. But you have to note two things:

    1.) The macro operator must be executed before/ the read operator. You can check and change the execution order within the process tab using the button displaying an up-down-arrow with an questionmark.

    2.) Metadata handling does not work anymore as the read operator does not know the filename before the process is run.

    Best regards,
    chero

  • CrazyTCrazyT Member Posts: 8 Contributor II
    Thanks for your reply, i was allways wondering, because i thought that you would need to link the macros as input.
    (but there is no input for data retrievers of course)
    Well I'm kinda new to this program, but i start to like it because its very useful.
    But now I'm having another problem, because my filenames aren't just with numbers but with dates.

    Is there a way to get the current date within rapidminer(with a macro)?
    I looked nearly everywhere but can't find any info about that.

    Only thing i found was something about adjusting a date in a column wich can be handy later.
    I also found the operator for generating data and for creating a data-column wich i thought could be used to create a column that contains the current date.
    Sadly i was wrong :(.

    Btw:
    Is there somewhere an overview of the functions you can use in macros?
    I found that page http://rapid-i.com/wiki/index.php?title=Generate_Macro , sadly there aren't any date functions.
    Are that the only functions that are supported at the moment?


    EDIT:
    http://rapid-i.com/wiki/index.php?title=First_Steps_With_RapidMiner#Parameter_Macros
    Alright, %{t} is what i need i guess.
    Sadly it outputs something like this:
    2010_06_26-AM_10_16_18
    But i need to get a value like:
    2010-06-26
    Anybody has any idea, how to format the value of a date inside a macro?

    Edit again - the solution:
    Alright, i solved the last problem myself.
    The example below takes the system date and subtracts 2 days from the current date.
    To view the content of the macro i added the column as output.(currdate)

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <process version="5.0">
     <context>
       <input/>
       <output/>
       <macros/>
     </context>
     <operator activated="true" class="process" compatibility="5.0.8" expanded="true" name="Process">
       <process expanded="true" height="476" width="480">
         <operator activated="true" class="generate_data" compatibility="5.0.8" expanded="true" height="60" name="Generate Data (2)" width="90" x="45" y="30">
           <parameter key="number_examples" value="1"/>
           <parameter key="number_of_attributes" value="1"/>
         </operator>
         <operator activated="true" class="set_macro" compatibility="5.0.8" expanded="true" height="76" name="Set Macro (3)" width="90" x="202" y="21">
           <parameter key="macro" value="added_days"/>
           <parameter key="value" value="-2"/>
         </operator>
         <operator activated="true" class="generate_attributes" compatibility="5.0.8" expanded="true" height="76" name="Generate Attributes (2)" width="90" x="45" y="120">
           <list key="function_descriptions">
             <parameter key="date" value="&quot;%{t}&quot;"/>
           </list>
         </operator>
         <operator activated="true" class="select_attributes" compatibility="5.0.8" expanded="true" height="76" name="Select Attributes" width="90" x="61" y="253">
           <parameter key="attribute_filter_type" value="subset"/>
           <parameter key="attributes" value="att1|label"/>
           <parameter key="invert_selection" value="true"/>
         </operator>
         <operator activated="true" class="nominal_to_date" compatibility="5.0.8" expanded="true" height="76" name="Nominal to Date (2)" width="90" x="45" y="390">
           <parameter key="attribute_name" value="date"/>
           <parameter key="date_format" value="y_M_d-a_h_m_s"/>
         </operator>
         <operator activated="true" class="adjust_date" compatibility="5.0.8" expanded="true" height="76" name="Adjust Date (2)" width="90" x="179" y="390">
           <parameter key="attribute_name" value="date"/>
           <list key="adjustments">
             <parameter key="%{added_days}" value="2"/>
           </list>
         </operator>
         <operator activated="true" class="date_to_nominal" compatibility="5.0.8" expanded="true" height="76" name="Date to Nominal (2)" width="90" x="246" y="120">
           <parameter key="attribute_name" value="date"/>
           <parameter key="date_format" value="yyyy-MM-dd"/>
         </operator>
         <operator activated="true" class="extract_macro" compatibility="5.0.8" expanded="true" height="60" name="Extract Macro (2)" width="90" x="313" y="255">
           <parameter key="macro" value="currdate"/>
           <parameter key="macro_type" value="data_value"/>
           <parameter key="attribute_name" value="date"/>
           <parameter key="example_index" value="1"/>
         </operator>
         <operator activated="true" class="generate_attributes" compatibility="5.0.8" expanded="true" height="76" name="Generate Attributes" width="90" x="380" y="120">
           <list key="function_descriptions">
             <parameter key="currdate" value="&quot;%{currdate}&quot;"/>
           </list>
         </operator>
         <connect from_op="Generate Data (2)" from_port="output" to_op="Set Macro (3)" to_port="through 1"/>
         <connect from_op="Set Macro (3)" from_port="through 1" to_op="Generate Attributes (2)" to_port="example set input"/>
         <connect from_op="Generate Attributes (2)" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
         <connect from_op="Select Attributes" from_port="example set output" to_op="Nominal to Date (2)" to_port="example set input"/>
         <connect from_op="Nominal to Date (2)" from_port="example set output" to_op="Adjust Date (2)" to_port="example set input"/>
         <connect from_op="Adjust Date (2)" from_port="example set output" to_op="Date to Nominal (2)" to_port="example set input"/>
         <connect from_op="Date to Nominal (2)" from_port="example set output" to_op="Extract Macro (2)" to_port="example set"/>
         <connect from_op="Extract Macro (2)" from_port="example set" to_op="Generate Attributes" to_port="example set input"/>
         <connect from_op="Generate Attributes" from_port="example set output" to_port="result 1"/>
         <portSpacing port="source_input 1" spacing="0"/>
         <portSpacing port="sink_result 1" spacing="0"/>
         <portSpacing port="sink_result 2" spacing="0"/>
       </process>
     </operator>
    </process>
Sign In or Register to comment.