"Limit for columns of pivot table"

CrazyT

Hello there,

I'm really new to this program (RapidMiner 5.0.008) and I'm just using it privately.
I think there seems to be a problem with the pivot-operator.
If you use more than 83 columns, it looks like it allways returns just 1 column (Amount).
For some reason it doesn't do the thing for a column amount less than 84.
So is this just a limit?
If yes, why does there no error message or warning appear?
Is there any workaround for this limitation?

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" breakpoints="before" class="process" compatibility="5.0.0" expanded="true" name="Process">
    <process expanded="true" height="391" width="915">
      <operator activated="true" class="generate_transaction_data" compatibility="5.0.0" expanded="true" height="60" name="Generate Transaction Data" width="90" x="4" y="113">
        <parameter key="number_transactions" value="100000"/>
        <parameter key="number_customers" value="5000"/>
        <parameter key="number_items" value="84"/>
        <parameter key="number_clusters" value="1"/>
      </operator>
      <operator activated="true" class="pivot" compatibility="5.0.0" expanded="true" height="76" name="Pivot" width="90" x="179" y="210">
        <parameter key="group_attribute" value="Id"/>
        <parameter key="index_attribute" value="Item"/>
      </operator>
      <operator activated="true" class="replace_missing_values" compatibility="5.0.0" expanded="true" height="94" name="Replace Missing Values" width="90" x="246" y="75">
        <parameter key="default" value="zero"/>
        <list key="columns"/>
      </operator>
      <operator activated="true" class="numerical_to_binominal" compatibility="5.0.0" expanded="true" height="76" name="Numerical to Binominal" width="90" x="380" y="75"/>
      <operator activated="true" class="fp_growth" compatibility="5.0.0" expanded="true" height="76" name="FP-Growth" width="90" x="447" y="210"/>
      <operator activated="true" class="create_association_rules" compatibility="5.0.0" expanded="true" height="60" name="Create Association Rules" width="90" x="581" y="210">
        <parameter key="min_confidence" value="0.1"/>
      </operator>
      <connect from_op="Generate Transaction Data" from_port="output" to_op="Pivot" to_port="example set input"/>
      <connect from_op="Pivot" from_port="example set output" to_op="Replace Missing Values" to_port="example set input"/>
      <connect from_op="Replace Missing Values" from_port="example set output" to_op="Numerical to Binominal" to_port="example set input"/>
      <connect from_op="Numerical to Binominal" from_port="example set output" to_op="FP-Growth" to_port="example set"/>
      <connect from_op="Numerical to Binominal" from_port="original" to_port="result 2"/>
      <connect from_op="FP-Growth" from_port="frequent sets" to_op="Create Association Rules" to_port="item sets"/>
      <connect from_op="Create Association Rules" from_port="rules" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

Thanks for your answers. 
(If some of the sentence sound weird, its because english is not my native language)


Edit:
I just noticed that there seem to exist a bug-entry for that:

http://bugs.rapid-i.com/show_bug.cgi?id=245

Still wonder if there is a way around that problem.

Edit again (informations for the developers):
Alright ... i modified the source in eclipse to make that operator work in the way i need it.
Thats what i changed in the file "Example2AttributePivoting.java" as a temporary fix:
  Modified this line:

			constantAttributeValues = true;

  into that:

			constantAttributeValues = false;

I know that this line was there for a reason ... but when i tried to debug it i found out that somehow the condition at:

						if (lastExample.getValue(attribute) != example.getValue(attribute)) {
							constantAttributeValues = false;
							continue;
						}

never gets fired for i=1 (column at i=1 should be my Amount-attribute).
(its weird cos this only happens for a column-amount/item-amount > 83
Not to mention that this line seems to check if the content of the Amount column changed.
Although my Amount column does not even change at <=83  - its allways "1")

land

Hi,
thank you for your investigations regarding this bug. I will attach a link to this thread on the respective bug.

Greetings,
  Sebastian

fischer

Hi,

we now have a parameter to switch pivoting for constant columns on and off.

Cheers,
Simon