RapidMiner

Write Database overwrite mode

Contributor II

Write Database overwrite mode

Hi,

We are trying to load data from csv into IBM Big SQL table using 'Write Database', but its failing.
Below are various options we tried:
1. 'overwrite mode' set to 'append' to insert data into table for first, it fails.
SQLCODE=-206, SQLSTATE=42703, SQLERRMC=COLUMN_1, DRIVER=3.71.22.
The JDBC driver has thrown an SQLException.

2. 'overwrite mode' set to 'overwrite', it inserts the data but changes the table structure and priviledges.
After executing RapidMiner process it inserts data into table but select query for individual column fails and gives below error:
SELECT COLUMN_1 FROM TABLE_1

SQL0206N "COLUMN_1" is not valid in the context where it is used.
SQLSTATE=42703

7 REPLIES
Moderator

Re: Write Database overwrite mode

This looks like a possible JDBC driver issue. Did you correctly configure the JDBC driver ?
RMStaff

Re: Write Database overwrite mode

Hi deepak_talim,

 

The 'append' mode is more or less the equivalent to an INSERT statement. Thus, the table and its columns have to be created upfront.

The 'overwrite' mode is more or less the equivalent to DROP IF EXISTS followed by CREATE TABLE and INSERT. Thus, the new table only consists of the data you wrote in your last process execution.

 

The error message of your second attempt is explained in the IBM Knowledge Base entry https://www.ibm.com/support/knowledgecenter/en/SSEPGG_9.7.0/com.ibm.db2.luw.messages.sql.doc/doc/msq... and states that the Column does not exist in the table you selected the data from.

 

Hope this helps,

Edin

Contributor II

Re: Write Database overwrite mode

Hi Thomas,

 

We are using following jdbc drivers : db2jcc.jar & db2jcc4.jar (db2_db2driver_for_jdbc_sqlj_v11.1).

 

Can you please specify which drivers are compatible with RapidMiner ver 7.2 and Big SQL ver 4.2

 

 

Thanks

Deepak T

Contributor II

Re: Write Database overwrite mode

Hi Edin,

 

If "'overwrite' mode is more or less the equivalent to DROP IF EXISTS followed by CREATE TABLE and INSERT" then after creating new table I should be able to query the table specifying the columns "created" in the SELECT but getting above mentioned error.

Can you please help to understand why am getting the error?

 

Thanks

Deepak T

RMStaff

Re: Write Database overwrite mode

Hi Deepak,

 

Which operator did you use? Execute SQL or Read Database?

While the first can execute data manipulation queries and does not deliver any output (i.e. no data is fetched by RapidMiner) the latter is used to read data.

 

What output did you get from

SELECT * FROM TABLE

Best regards,

Edin

Contributor II

Re: Write Database overwrite mode

Hi,

We were able to solve the issue, it was related the way table is created in Big SQL.

 

If we create table WITHOUT 'quotes' / "" in table name or column name as below, it gives error once data is loaded using RapidMiner.
ex: create table dbo.table_test ( column_1 varchar(5), column_2 varchar(5))

 

If we create table WITH 'quotes' / "" in table name or column name as below, it works without any error even after data is loaded using RapidMiner.
ex: create table "dbo"."table_test" ( "column_1" varchar(5), "column_2" varchar(5))

 

For data load we can use 'overwrite' or 'overwrite first, append later', it works.

 

Thanks & Regards,

Deepak Talim

Highlighted
RMStaff

Re: Write Database overwrite mode

Great to hear that your problem is solved Smiley Happy

and Thank you for posting your solution!

 

Best,

Edin