RapidMiner

RapidMiner

How to generate attributes using attributes from two datasets (sources)

Contributor II

How to generate attributes using attributes from two datasets (sources)

Hello! I am new with rapidminer, and i am taking the first steps with ETL processes. Now, i have found a problem that i don't know how to solve it. In particular, I am looking for a way to generating new attributes applying function expressions where I need to use attributes from two database tables (dataset1, dataset2), where one of the tables (dataset2) has attributes that i need to use to calculate new attributes in the other table (dataset1).

Tables have different number of rows, and need to calculate a particular value of att1 in dataset1 to apply as a constant to calculate new attribute.

dataset1

                    att1 att2 att3
        row1    1.1  1.2  2.3
        row2    1      2.2 3.3
        row3    1      2.4  5.3

dataset2

                    att1 att2 att3
        row1    1.1  2    3
        row2    1.2  2    3
        row3    1.3  2    3
        row4    1.4  2    3
        row5    1.6  2    3
        row6    1.7  2    3


Using Generate attributes

new attribute (dataset1)

dataset1.att4 = dataset1.att1 * dataset2.att1


Is it possible to use attributes from other datasets to calculate new attributes in other dataset?

Best regards,

alberto

2 REPLIES
Regular Contributor

Re: How to generate attributes using attributes from two datasets (sources)

Hello,

Based on your example, what should be the result?
Should I simply discard
      row4    1.4  2    3
        row5    1.6  2    3
        row6    1.7  2    3

Best regards,

Wessel

Edit:

See process below, with join and generate attributes.
Regular Contributor

Re: How to generate attributes using attributes from two datasets (sources)

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.3.013">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.3.013" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="generate_data" compatibility="5.3.013" expanded="true" height="60" name="Generate Data" width="90" x="228" y="101">
        <parameter key="number_examples" value="3"/>
      </operator>
      <operator activated="true" class="generate_id" compatibility="5.3.013" expanded="true" height="76" name="Generate ID" width="90" x="453" y="122"/>
      <operator activated="true" class="generate_data" compatibility="5.3.013" expanded="true" height="60" name="Generate Data (2)" width="90" x="223" y="203">
        <parameter key="number_examples" value="5"/>
      </operator>
      <operator activated="true" class="generate_id" compatibility="5.3.013" expanded="true" height="76" name="Generate ID (2)" width="90" x="415" y="272"/>
      <operator activated="true" class="rename_by_replacing" compatibility="5.3.013" expanded="true" height="76" name="Rename by Replacing" width="90" x="547" y="420">
        <parameter key="replace_what" value="$"/>
        <parameter key="replace_by" value="_2"/>
      </operator>
      <operator activated="true" class="join" compatibility="5.3.013" expanded="true" height="76" name="Join" width="90" x="573" y="216">
        <list key="key_attributes"/>
      </operator>
      <operator activated="true" class="generate_attributes" compatibility="5.3.013" expanded="true" height="76" name="Generate Attributes" width="90" x="735" y="162">
        <list key="function_descriptions">
          <parameter key="att1*att1_2" value="att1*att1_2"/>
        </list>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="Generate ID" to_port="example set input"/>
      <connect from_op="Generate ID" from_port="example set output" to_op="Join" to_port="left"/>
      <connect from_op="Generate Data (2)" from_port="output" to_op="Generate ID (2)" to_port="example set input"/>
      <connect from_op="Generate ID (2)" from_port="example set output" to_op="Rename by Replacing" to_port="example set input"/>
      <connect from_op="Rename by Replacing" from_port="example set output" to_op="Join" to_port="right"/>
      <connect from_op="Join" from_port="join" to_op="Generate Attributes" to_port="example set input"/>
      <connect from_op="Generate Attributes" from_port="example set output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>