"Problems with generating Association Rules"

riddler16riddler16 Member Posts: 2 Contributor I
edited June 2019 in Help
Hi all i´m new here and i have a question about the FP-Algorithms, Association Rules and espacially about buffering :)

My whole RM process is very simple. I read about 2000 transactions from a csv file. Then there is a preprocessing step with a nominal2binominal operator. The output goes to the fp-growth and after fp-growth to the AssociationRulesGenerator.
Everything works fine...till i choose a low support. (about 0.2).
I get the error message that i have to less memory and that i should update it or try the cached database operator. I have already tried it with more then 10GB Ram and still the same :(
My question is, if there is an chance to get these rules? Maybe by caching the results or writing into a file or something like this?
The Problem is not the fp-growth algorithm. At least i think so, because the status bar on the bottom of RM tells me after some seconds, that the fp-growth calculation is over and now the generation of the association rules beginns

Here is the xml-file:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.0.0" expanded="true" name="Root">
    <parameter key="logverbosity" value="warning"/>
    <process expanded="true" height="584" width="918">
      <operator activated="true" class="read_csv" compatibility="5.0.10" expanded="true" height="60" name="Read CSV" width="90" x="29" y="23">
        <parameter key="file_name" value="\output.csv"/>
        <parameter key="encoding" value="windows-1252"/>
        <parameter key="skip_comments" value="false"/>
        <parameter key="use_first_row_as_attribute_names" value="false"/>
        <list key="data_set_meta_data_information">
          <parameter key="0" value="attribute_0.true.1.regular"/>
          ...
          <parameter key="30" value="attribute_30.false.1.regular"/>
        </list>
        <parameter key="attribute_names_already_defined" value="true"/>
      </operator>
      <operator activated="true" class="subprocess" compatibility="5.0.0" expanded="true" height="76" name="Preprocessing" width="90" x="180" y="30">
        <process expanded="true" height="605" width="950">
          <operator activated="true" class="nominal_to_binominal" compatibility="5.0.0" expanded="true" height="94" name="Nominal2Binominal" width="90" x="497" y="30">
            <parameter key="transform_binominal" value="true"/>
          </operator>
          <connect from_port="in 1" to_op="Nominal2Binominal" to_port="example set input"/>
          <connect from_op="Nominal2Binominal" from_port="example set output" to_port="out 1"/>
          <portSpacing port="source_in 1" spacing="0"/>
          <portSpacing port="source_in 2" spacing="0"/>
          <portSpacing port="sink_out 1" spacing="0"/>
          <portSpacing port="sink_out 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="fp_growth" compatibility="5.0.0" expanded="true" height="76" name="FPGrowth" width="90" x="313" y="30">
        <parameter key="find_min_number_of_itemsets" value="false"/>
        <parameter key="min_support" value="0.1"/>
      </operator>
      <operator activated="true" class="create_association_rules" compatibility="5.0.0" expanded="true" height="60" name="AssociationRuleGenerator" width="90" x="447" y="75">
        <parameter key="min_confidence" value="0.7"/>
      </operator>
      <connect from_op="Read CSV" from_port="output" to_op="Preprocessing" to_port="in 1"/>
      <connect from_op="Preprocessing" from_port="out 1" to_op="FPGrowth" to_port="example set"/>
      <connect from_op="FPGrowth" from_port="example set" to_port="result 2"/>
      <connect from_op="FPGrowth" from_port="frequent sets" to_op="AssociationRuleGenerator" to_port="item sets"/>
      <connect from_op="AssociationRuleGenerator" from_port="rules" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="36"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>
Thanks a lot for any ideas.

Answers

  • riddler16riddler16 Member Posts: 2 Contributor I
    Hey,

    I just recognised that the problem is not the association rules generator. It is the WriteIntoFile-Operator. I just run the same configuration but without any output and it seems to work.
    So now i'm searching for any other possiblities to manage this problem because of course i need the output :)
    The problem is that i don´t know if the file size is getting to big for the OS or the filesystem have to less memory to save this file.
    Maybe somebody here has a suggestion how to deal with it?

Sign In or Register to comment.