Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Stream Database Error with Postgres 9
Hello,
I am facing an issue when trying to stream a very large table in Rapid Miner Studio 7 from Postgres 9.
It appears that Rapid Miner is creating the secondary index mapping table, but it always errors out with the following text. Process failed: Database error occurred: ERROR: syntax error at or near "IDENTITY"
Position: 76
The table does have a compound primary key like the following, perhaps that is causing issues?
CONSTRAINT pk_nc9_fullfile PRIMARY KEY (field1, field2)
This table that I am trying query is very large, too large to load the entire thing into memory using the Read Database or Retrieve data operators. Has anyone seen this behavior before?
Thanks
I am facing an issue when trying to stream a very large table in Rapid Miner Studio 7 from Postgres 9.
It appears that Rapid Miner is creating the secondary index mapping table, but it always errors out with the following text. Process failed: Database error occurred: ERROR: syntax error at or near "IDENTITY"
Position: 76
The table does have a compound primary key like the following, perhaps that is causing issues?
CONSTRAINT pk_nc9_fullfile PRIMARY KEY (field1, field2)
This table that I am trying query is very large, too large to load the entire thing into memory using the Read Database or Retrieve data operators. Has anyone seen this behavior before?
Thanks
Tagged:
0
Answers
Stream Database needs a special index structure. It can't work with your table as it is now.
The error message is coming from the futile attempt to create a new index field in your table. But this uses non-standard (probably MySQL) syntax that PostgreSQL doesn't accept.
Try to find combinations from your compound index or other factors that allow you to do the batching yourself. Then you could use the Loop Values operator to execute the query for each batch.
It would appear this operator does not work well with postgres
I still don't understand why I have to do this, as it defeats the purpose of having rapid miner make a new key mapping table or adding the itself, but it's a solution for now.
Thanks for looking.