Options
CachedDatabaseExampleSource Error
Maven
Hi, I would like to read data from my database, then convert Nominal2Binominal and apply FPGrowth. The Problem is that there is not enough memory for the Nominal2Binominal Operator. So I put it into a BatchProcessing Operator and write the results into a table. I have to use the CachedDatabaseExampleSource for reading because there is also not enough memory to read from the table. But when i start the Process I get the following Error:
Error in: CachedDatabaseExampleSource (CachedDatabaseExampleSource) Database error occurred: FEHLER: Fehler »Syntaxfehler« bei »AUTO_INCREMENT
This is the Process. I am using PostgreSQL as DBMS.
<operator name="Root" class="Process" expanded="yes">
<operator name="DatabaseExampleSource" class="DatabaseExampleSource">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j"/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="query" value="SE"/>
</operator>
<operator name="BatchProcessing" class="BatchProcessing" expanded="yes">
<parameter key="batch_size" value="5000"/>
<operator name="Nominal2Binominal" class="Nominal2Binominal">
</operator>
<operator name="DatabaseExampleSetWriter" class="DatabaseExampleSetWriter">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="table_name" value="sell_temp"/>
<parameter key="overwrite_mode" value="append"/>
</operator>
</operator>
<operator name="OperatorChain" class="OperatorChain" expanded="yes">
<operator name="CachedDatabaseExampleSource" class="CachedDatabaseExampleSource">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j"/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="table_name" value="sell_temp"/>
<parameter key="recreate_index" value="true"/>
</operator>
<operator name="DatabaseExampleSource (2)" class="DatabaseExampleSource" activated="no">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j"/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="query" value="SELECT * from public.sell_temp"/>
</operator>
<operator name="FPGrowth" class="FPGrowth">
<parameter key="find_min_number_of_itemsets" value="false"/>
<parameter key="min_support" value="0.0050"/>
</operator>
<operator name="AssociationRuleGenerator" class="AssociationRuleGenerator">
<parameter key="min_confidence" value="0.0050"/>
</operator>
</operator>
</operator>
Does anybody know how to correct the error or has another solution for my problem ?
Error in: CachedDatabaseExampleSource (CachedDatabaseExampleSource) Database error occurred: FEHLER: Fehler »Syntaxfehler« bei »AUTO_INCREMENT
This is the Process. I am using PostgreSQL as DBMS.
<operator name="Root" class="Process" expanded="yes">
<operator name="DatabaseExampleSource" class="DatabaseExampleSource">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j"/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="query" value="SE"/>
</operator>
<operator name="BatchProcessing" class="BatchProcessing" expanded="yes">
<parameter key="batch_size" value="5000"/>
<operator name="Nominal2Binominal" class="Nominal2Binominal">
</operator>
<operator name="DatabaseExampleSetWriter" class="DatabaseExampleSetWriter">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="table_name" value="sell_temp"/>
<parameter key="overwrite_mode" value="append"/>
</operator>
</operator>
<operator name="OperatorChain" class="OperatorChain" expanded="yes">
<operator name="CachedDatabaseExampleSource" class="CachedDatabaseExampleSource">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j"/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="table_name" value="sell_temp"/>
<parameter key="recreate_index" value="true"/>
</operator>
<operator name="DatabaseExampleSource (2)" class="DatabaseExampleSource" activated="no">
<parameter key="database_system" value="PostgreSQL"/>
<parameter key="database_url" value="j"/>
<parameter key="username" value="postgres"/>
<parameter key="password" value="GGNSbFYYAWMTO84DfmHmdQ=="/>
<parameter key="query" value="SELECT * from public.sell_temp"/>
</operator>
<operator name="FPGrowth" class="FPGrowth">
<parameter key="find_min_number_of_itemsets" value="false"/>
<parameter key="min_support" value="0.0050"/>
</operator>
<operator name="AssociationRuleGenerator" class="AssociationRuleGenerator">
<parameter key="min_confidence" value="0.0050"/>
</operator>
</operator>
</operator>
Does anybody know how to correct the error or has another solution for my problem ?
0
Answers
I guess the problem is, that the CachedDatabaseExampleSource needs an autoincremented index key for accessing the table. At least I guess so. If you wish, I could ask my college who implemented it.
But anyway I would recommend another approach:
Use create view on the Nominal2Binominal. It will save the memory and computes the values each time the value is accessed. But since FPGrowth will read it only once, it does not compute it more often than without view.
Another possible error source might be, if the Nominal2Binominal will create more then around 1000 columns: No current database system I know handles much more than 1000 columns.
Greetings,
Sebastian